The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews
Open accessAbstract
Objective Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR).Study design and setting We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times.Results The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete.Conclusions AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“two independent authors assessed the quality of the systematic reviews and meta-analyses using the AMSTAR and AMSTAR+ tool [53][54][55].”
From Exercise as Medicine for Mental and Substance Use Disorders: A Meta-review of the Benefits for Neuropsychiatric and Cognitive Outcomes (2020) · cites this paper as [53]
The cited source says what the claim attributes to it.
In the citing paper
“The methodological quality of these reviews was examined using the AMSTAR tool [38].”
From The effectiveness of mindfulness-based interventions on anxiety disorders. A systematic meta-review (2020) · cites this paper as [38]
In the citing paper
“The validity and reliability of the AMSTAR has been established in previous studies.19,20”
From Quality assessment of systematic reviews on the efficacy of oral appliance therapy for adult and pediatric sleep-disordered breathing (2016) · cites this paper as [19]
In the citing paper
“The instrument is a reliable and valid measure for the evaluation of the methodological quality of systematic reviews and has proven good face and content validity.20”
From Fiber burden and asbestos-related diseases: an umbrella review (2022) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Self-citationAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Users' Guides to the Medical Literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 3,083
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,312
Systematic Reviews in Health Care. Meta-analysis in Context.
Kleijnen, J., & Antes, G. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.3.697 · Cited by 1,374
Research works printed but not linked: 12
- Counsell, 1996, Different patches in carotoid surgery, Cochrane Library
- Daya, 1996, Comparison of FSH and HMG in IVF, Cochrane Library
- Duley, 1996, Anticonvulsants for pre-eclampsia, Cochrane Library
- Graves, 1996, Malaria vaccines, Cochrane Library
- Hodnett, 1996, Alternative versus conventional delivery settings, Cochrane Library
- Hopfenmuller, 1994, Nackweis der therapeutischen Wirksamkeit eines Ginkgo biloba-Spezial extrakes: Meta-Analyse von 11 klinischen Studien bei Patienten mit Hirnleistungsstorungen im Alter, Arzneimittel-Forschung, 44, 1005
- Kramer, 1996, Maternal antigen avoidance as lactation, Cochrane Library
- McGrath, 1996, Tardive dyskinesia and benzodiazepines, Cochrane Library
- Renfrew, 1996, Breastfeeding and discharge times, Cochrane Library
- Renfrew, 1996, Breastfeeding and early contact, Cochrane Library
- Soares, 1996, Tardive dyskinesia and GABA agonist drugs, Cochrane Library
- Bland, 1987, Statistical methods for assessing agreement between measurements, Biochim Clin, 11, 399
Cited by
Sorted by citations
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Self-citationQaseem, A., Wilt, T. J., McLean, R. M., Forciea, M. A., & for the Clinical Guidelines Committee of the American College of Physicians* (2017) · Annals of Internal Medicine · doi:10.7326/m16-2367 · Cited by 3,234
Risks of all-cause and suicide mortality in mental disorders: a meta-review
Chesney, E., Goodwin, G. M., & Fazel, S. (2014) · World Psychiatry · doi:10.1002/wps.20128 · Cited by 2,115
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
Nurturing care: Promoting early childhood development
Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., Perez-Escamilla, R., Rao, N., Ip, P., Fernald, L. C. H., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31390-3 · Cited by 1,796
Metadata
- Authors
- Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M.
- Affiliations
- Amsterdam Public Health (NL)
- Institute of Population and Public Health (CA)
- University of Ottawa (CA)
- Institute for Clinical Evaluative Sciences (CA)
- Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Shea, Beverley J.
- ORCID
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2009-10
- Volume
- 62
- Issue
- 10
- Pages
- 1013-1020
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2073644162
- Cited by
- 1,755
- References
- 47
- Citations by year
- 2026: 47 · 2025: 58 · 2024: 64 · 2023: 100 · 2022: 79 · 2021: 148 · 2020: 140 · 2019: 158 · 2018: 168 · 2017: 216 · 2016: 149 · 2015: 121 · 2014: 108 · 2013: 83 · 2012: 49
- FWCI
- 2.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- AMSTAR, methodological quality assessment, systematic reviews, inter-rater reliability, construct validity
- MeSH terms
- Feasibility Studies, Humans, Quality Control, Research Design, Review Literature as Topic, Reproducibility of Results, Observer Variation, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
