The DOI matches its Crossref record; 75 of the 76 references it cites were found in a registry or an index.
External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Open accessAbstract
BACKGROUND: Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews. METHODOLOGY: AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently. CONCLUSIONS: The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is a revised version of the original AMSTAR checklist [18], which is a validated measure of systematic review quality [19].”
From Implementation of Telemental Health Services Before COVID-19: Rapid Umbrella Review of Systematic Reviews (2021) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The validity and reliability of the AMSTAR has been established in previous studies.19,20”
From Quality assessment of systematic reviews on the efficacy of oral appliance therapy for adult and pediatric sleep-disordered breathing (2016) · cites this paper as [20]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
6 retracted
Eradication of Helicobacter pylori for non-ulcer dyspepsia
Moayyedi, P., Soo, S., Deeks, J., Delaney, B., Harris, A., Innes, M., Oakes, R., Wilson, S., Roalfe, A., Bennett, C., et al. (2003) · The Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002096 · Cited by 371
RetractedEradication of Helicobacter pylori for non-ulcer dyspepsia
Moayyedi, P., Soo, S., Deeks, J. J., Delaney, B., Harris, A., Innes, M., Oakes, R., Wilson, S., Roalfe, A., Bennett, C., et al. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002096.pub4 · Cited by 195 · cited 1×
“The topics of the reviews ranged across the spectrum of GI problems like dyspepsia, gastroesophageal reflux disease (GERD), peptic ulcer disease (PUD), and also GI drug interventions such as H2 receptor antagonists and proton pump inhibitors [9–50].”
BackgroundRetractedInitial management strategies for dyspepsia
Delaney, B., Moayyedi, P., & Forman, D. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001961 · Cited by 181
RetractedPharmacological interventions for non-ulcer dyspepsia
Moayyedi, P., Soo, S., Deeks, J. J., DELANEY, B., Innes, M., & Forman, D. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001960 · Cited by 114
RetractedPharmacological interventions for non-ulcer dyspepsia
Moayyedi, P., Shelly, S., Deeks, J. J., Delaney, B., Innes, M., & Forman, D. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001960.pub3 · Cited by 99 · cited 1×
“The topics of the reviews ranged across the spectrum of GI problems like dyspepsia, gastroesophageal reflux disease (GERD), peptic ulcer disease (PUD), and also GI drug interventions such as H2 receptor antagonists and proton pump inhibitors [9–50].”
BackgroundRetractedEradication of Helicobacter pylori for non-ulcer dyspepsia
Moayyedi, P., Soo, S., Deeks, J. J., Delaney, B. C., Harris, A. W., Innes, M. A., Oakes, R., Wilson, S., Roalfe, A. K., Bennett, C., et al. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002096.pub2 · Cited by 60
RetractedSTATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 2×
“Kappa values of less than 0 rate as less than chance agreement; 0.01–0.20 slight agreement; 0.21– 0.40 fair agreement; 0.41–0.60 moderate agreement; 0.61–0.80 substantial agreement; and 0.81–0.99 almost perfect agreement [52], [57].”
BackgroundMethodInfluentialMeasuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Research works printed but not linked: 3
- EF Trépanier, 1998, Meta-analysis of H pylori (HP) eradication rates in patients with duodenal ulcer (DU)., Can J Clin Pharmacol., 5(1), 67
- AD Oxman, 1991, Validation of an index of the quality of review articles., J Clin Epidemiol, 44(11), 1271, 10.1016/0895-4356(91)90160-B · DOI · Paper page
- JM Bland, 1987, Statistical methods for assessing agreement between measurement., Biochimica Clinica, 11, 399
Cited by
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Self-citationShea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
“We found very good correlations on the total scores [6].”
Self-citationSedentary behavior and health outcomes: an overview of systematic reviews.
Rezende, L. F. M. D., Rodrigues Lopes, M., Rey-López, J. P., Matsudo, V. K. R., & Luiz, O. D. C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105620 · Cited by 965
“[44,45] AMSTAR contains 11-items to appraise the methodological aspects of the systematic reviews.”
BackgroundTen simple rules for conducting umbrella reviews
Fusar-Poli, P., & Radua, J. (2018) · Evidence Based Mental Health · doi:10.1136/ebmental-2018-300014 · Cited by 724
“In the absence of specific guidelines for quality assessment in umbrella reviews, A Measurement Tool to Assess Systematic Reviews, a validated instrument 21 22 can be used.”
MethodA systematic review of industrial wastewater management: Evaluating challenges and enablers
Singh, B. J., Chakraborty, A., & Sehgal, R. (2023) · Journal of Environmental Management · doi:10.1016/j.jenvman.2023.119230 · Cited by 610
“Another study by Shea et al. (2007) highlighted the importance of evaluating the reporting quality of the studies to ensure that all relevant information is reported accurately.”
Background
Metadata
- Authors
- Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M.
- Affiliations
- Amsterdam Public Health (NL)
- Vrije Universiteit Amsterdam (NL)
- University of Ottawa (CA)
- Universidad Autónoma de Guerrero (MX)
- Academia Nacional de Medicina (AR)
- Canada's Drug Agency (CA)
- Corresponding author
- Shea, Beverley J.
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2007-12-26
- Volume
- 2
- Issue
- 12
- Pages
- e1350
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2131785
- OpenAlex
- W2024037973
- Cited by
- 569
- References
- 76
- Citations by year
- 2026: 6 · 2025: 20 · 2024: 20 · 2023: 25 · 2022: 27 · 2021: 38 · 2020: 33 · 2019: 37 · 2018: 61 · 2017: 58 · 2016: 50 · 2015: 38 · 2014: 42 · 2013: 37 · 2012: 25
- FWCI
- 2.17
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- AMSTAR, methodological quality assessment, external validation, interobserver agreement, construct validity, reliability statistics, gastroesophageal reflux disease, peptic ulcer, acid-related diseases
- MeSH terms
- Gastric Acid, Gastroesophageal Reflux, Humans, Peptic Ulcer, Meta-Analysis as Topic, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR). https://app.cochat.ai/papers/10.1371/journal.pone.0001350
- The paper
- Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007). External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR). PLoS ONE. https://doi.org/10.1371/journal.pone.0001350
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
