The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews
Open accessAbstract
Objective Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR).Study design and setting We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times.Results The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete.Conclusions AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“two independent authors assessed the quality of the systematic reviews and meta-analyses using the AMSTAR and AMSTAR+ tool [53][54][55].”
From Exercise as Medicine for Mental and Substance Use Disorders: A Meta-review of the Benefits for Neuropsychiatric and Cognitive Outcomes (2020) · cites this paper as [53]
The cited source says what the claim attributes to it.
In the citing paper
“The methodological quality of these reviews was examined using the AMSTAR tool [38].”
From The effectiveness of mindfulness-based interventions on anxiety disorders. A systematic meta-review (2020) · cites this paper as [38]
In the citing paper
“The validity and reliability of the AMSTAR has been established in previous studies.19,20”
From Quality assessment of systematic reviews on the efficacy of oral appliance therapy for adult and pediatric sleep-disordered breathing (2016) · cites this paper as [19]
In the citing paper
“The instrument is a reliable and valid measure for the evaluation of the methodological quality of systematic reviews and has proven good face and content validity.20”
From Fiber burden and asbestos-related diseases: an umbrella review (2022) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Self-citationAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Users' Guides to the Medical Literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 3,083
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,312
Systematic Reviews in Health Care. Meta-analysis in Context.
Kleijnen, J., & Antes, G. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.3.697 · Cited by 1,374
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,257
Whittington, C. J., Kendall, T., Fonagy, P., Cottrell, D., Cotgrove, A., & Boddington, E. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16043-1 · Cited by 1,006
Effects of Adjuvant Tamoxifen and of Cytotoxic Therapy on Mortality in Early Breast Cancer
Early Breast Cancer Trialists' Collaborative Group (1988) · New England Journal of Medicine · doi:10.1056/nejm198812293192601 · Cited by 932
Validation of an index of the quality of review articles
Oxman, A. D., & Guyatt, G. H. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90160-b · Cited by 788
Prevention of Venous Thromboembolism in General Surgical Patients
Clagett, G. P., & REISCH, J. S. (1988) · Annals of Surgery · doi:10.1097/00000658-198808000-00016 · Cited by 688
External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569
Self-citationDoes Supplementation of Diet With 'Fish Oil' Reduce Blood Pressure?
Appel, L. J. (1993) · Archives of Internal Medicine · doi:10.1001/archinte.153.12.1429 · Cited by 491
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478
Why review articles on the health effects of passive smoking reach different conclusions
Barnes, D. E. (1998) · JAMA · doi:10.1001/jama.279.19.1566 · Cited by 442
Evidence Favoring the Use of Anticoagulants in the Hospital Phase of Acute Myocardial Infarction
Chalmers, T. C., Matta, R. J., Smith, H., & Kunzler, A.-M. (1977) · New England Journal of Medicine · doi:10.1056/nejm197711172972004 · Cited by 423
Hughes, E. G., Fedorkow, D. M., Daya, S., Sagle, M. A., de Koppel, P. V., & Collins, J. A. (1992) · Fertility and Sterility · doi:10.1016/s0015-0282(16)55430-2 · Cited by 401
Crystalloid versus colloid fluid resuscitation: a meta-analysis of mortality.
Velanovich, V. (1989) · PubMed · doi:10.5555/uri:pii:003960608990010x · Cited by 254
Diversity of decision-making models and the measurement of interrater agreement
Uebersax, J. S. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.101.1.140 · Cited by 254
Pérez-Escamilla, R., Pollitt, E., Lönnerdal, B. L., & Dewey, K. G. (1994) · American Journal of Public Health · doi:10.2105/ajph.84.1.89 · Cited by 221
Magnesium sulphate and other anticonvulsants for women with pre-eclampsia
Duley, L., Gülmezoglu, A. M., & Henderson-Smart, D. J. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000025 · Cited by 211
Biondi-Zoccai, G. G. L., Lotrionte, M., Abbate, A., Testa, L., Remigi, E., Burzotta, F., Valgimigli, M., Romagnoli, E., Crea, F., & Agostoni, P. (2006) · BMJ · doi:10.1136/bmj.38693.516782.7c · Cited by 149
Challenges in Systematic Reviews That Assess Treatment Harms
Chou, R., & Helfand, M. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-12_part_2-200506211-00009 · Cited by 145
A methodologic framework for developing and selecting endpoints in clinical trials.
Tugwell, P., & Bombardier, C. (1983) · PubMed · Cited by 137
Treatments of preterm premature rupture of the membranes: A meta-analysis
Ohlsson, A. (1989) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(89)90306-2 · Cited by 132
Rudy, E. B., Lucke, J. F., Whitman, G. R., et al. (2001) · Cited by 121
Oxman, A. D., Schünemann, H. J., & Fretheim, A. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-20 · Cited by 116
Assessing the Quality of Reports of Systematic Reviews: The QUOROM Statement Compared to Other Tools
Shea, B., Dubé, C., & Moher, D. (2001) · Wiley eBooks · doi:10.1002/9780470693926.ch7 · Cited by 80
Self-citationSomatostatin v placebo in bleeding oesophageal varices: randomised trial and meta-analysis
Gotzsche, P. C., Gjørup, I. E., Bonnén, H., Brahe, N. E. B., Becker, U., & Burcharth, F. (1995) · BMJ · doi:10.1136/bmj.310.6993.1495 · Cited by 77
Antiplatelet or Anticoagulant Therapy after Coronary Artery Bypass Surgery
Henderson, W. G., Goldman, S., Copeland, J. G., Moritz, T. E., & Harker, L. A. (1989) · Annals of Internal Medicine · doi:10.7326/0003-4819-111-9-743 · Cited by 74
Quality of controlled clinical trials. The case of imaging ultrasound in obstetrics: a review
Thacker, S. B. (1985) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.1985.tb01346.x · Cited by 73
Fanning, J., Bennett, T. Z., & Hilgers, R.-D. (1992) · PubMed · Cited by 67
Hopfenmüller, W. (1994) · PubMed · Cited by 52
Hypertension intervention pooling project.
Kaufmann, P. G., HIPP Investigators, Jacob, R. G., Ewart, C. K., Chesney, M. A., Muenz, L. R., Doub, N., & Mercer, W. (1988) · Health Psychology · doi:10.1037/0278-6133.7.suppl.209 · Cited by 41
Relative Efficacy of Vasodilator Therapy in Chronic Congestive Heart Failure
Mulrow, C. D. (1988) · JAMA · doi:10.1001/jama.1988.03720230032026 · Cited by 40
Gent, M., & Roberts, R. S. (1986) · CHEST Journal · doi:10.1378/chest.89.5_supplement.396s · Cited by 40
Early versus delayed initiation of breastfeeding.
Renfrew, M. J., Lang, S. P., & Woolridge, M. W. (2000) · PubMed · doi:10.1002/14651858.cd000043 · Cited by 39
A meta-analysis of the use of amoxycillin-clavulanic acid in surgical prophylaxis
Wilson, A. P. R., Shrimpton, S. B., & Jaderberg, M. (1992) · Journal of Hospital Infection · doi:10.1016/s0195-6701(05)80003-9 · Cited by 28
Randomized Trials of Aminoglycoside Antibiotics: Quantitative Overview
Buring, J. E., Evans, D. A., Mayrent, S. L., et al. (1988) · Clinical Infectious Diseases · doi:10.1093/clinids/10.5.951 · Cited by 28
Postherpetic Neuralgia and Systemic Corticosteroid Therapy
Lycka, B. A. S. (1990) · International Journal of Dermatology · doi:10.1111/j.1365-4362.1990.tb04852.x · Cited by 26
Pavelka, R., & Salzer, H. (2010) · Gynecologic and Obstetric Investigation · doi:10.1159/000299661 · Cited by 8
Mulrow, C. D. (1988) · JAMA · doi:10.1001/jama.259.23.3422 · Cited by 3
Research works printed but not linked: 12
- Counsell, 1996, Different patches in carotoid surgery, Cochrane Library
- Daya, 1996, Comparison of FSH and HMG in IVF, Cochrane Library
- Duley, 1996, Anticonvulsants for pre-eclampsia, Cochrane Library
- Graves, 1996, Malaria vaccines, Cochrane Library
- Hodnett, 1996, Alternative versus conventional delivery settings, Cochrane Library
- Hopfenmuller, 1994, Nackweis der therapeutischen Wirksamkeit eines Ginkgo biloba-Spezial extrakes: Meta-Analyse von 11 klinischen Studien bei Patienten mit Hirnleistungsstorungen im Alter, Arzneimittel-Forschung, 44, 1005
- Kramer, 1996, Maternal antigen avoidance as lactation, Cochrane Library
- McGrath, 1996, Tardive dyskinesia and benzodiazepines, Cochrane Library
- Renfrew, 1996, Breastfeeding and discharge times, Cochrane Library
- Renfrew, 1996, Breastfeeding and early contact, Cochrane Library
- Soares, 1996, Tardive dyskinesia and GABA agonist drugs, Cochrane Library
- Bland, 1987, Statistical methods for assessing agreement between measurements, Biochim Clin, 11, 399
Cited by
Sorted by citations
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Self-citationQaseem, A., Wilt, T. J., McLean, R. M., Forciea, M. A., & for the Clinical Guidelines Committee of the American College of Physicians* (2017) · Annals of Internal Medicine · doi:10.7326/m16-2367 · Cited by 3,234
Risks of all-cause and suicide mortality in mental disorders: a meta-review
Chesney, E., Goodwin, G. M., & Fazel, S. (2014) · World Psychiatry · doi:10.1002/wps.20128 · Cited by 2,115
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
Nurturing care: Promoting early childhood development
Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., Perez-Escamilla, R., Rao, N., Ip, P., Fernald, L. C. H., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31390-3 · Cited by 1,796
Metadata
- Authors
- Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M.
- Affiliations
- Amsterdam Public Health (NL)
- Institute of Population and Public Health (CA)
- University of Ottawa (CA)
- Institute for Clinical Evaluative Sciences (CA)
- Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Shea, Beverley J.
- ORCID
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2009-10
- Volume
- 62
- Issue
- 10
- Pages
- 1013-1020
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2073644162
- Cited by
- 1,755
- References
- 47
- Citations by year
- 2026: 47 · 2025: 58 · 2024: 64 · 2023: 100 · 2022: 79 · 2021: 148 · 2020: 140 · 2019: 158 · 2018: 168 · 2017: 216 · 2016: 149 · 2015: 121 · 2014: 108 · 2013: 83 · 2012: 49
- FWCI
- 2.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- AMSTAR, methodological quality assessment, systematic reviews, inter-rater reliability, construct validity
- MeSH terms
- Feasibility Studies, Humans, Quality Control, Research Design, Review Literature as Topic, Reproducibility of Results, Observer Variation, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
