Efficient clinical evaluation of guideline quality: development and testing of a new tool
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Evaluating the methodological quality of clinical practice guidelines is essential before deciding which ones which could best inform policy or practice. One current method of evaluating clinical guideline quality is the research-focused AGREE II instrument. This uses 23 questions scored 1-7, arranged in six domains, which requires at least two independent testers, and uses a formulaic weighted domain scoring system. Following feedback from time-poor clinicians, policy-makers and managers that this instrument did not suit clinical need, we developed and tested a simpler, shorter, binary scored instrument (the iCAHE Guideline Quality Checklist) designed for single users. METHODS: Content and construct validity, inter-tester reliability and clinical utility were tested by comparing the new iCAHE Guideline Quality Checklist with the AGREE II instrument. Firstly the questions and domains in both instruments were compared. Six randomly-selected guidelines on a similar theme were then assessed by three independent testers with different experience in guideline quality assessment, using both instruments. Per guideline, weighted domain and total AGREE II scores were calculated, using the scoring rubric for three testers. Total iCAHE scores were calculated per guideline, per tester. The linear relationship between iCAHE and AGREE II scores was assessed using Pearson r correlation coefficients. Score differences between testers were assessed for the iCAHE Guideline Quality Checklist. RESULTS: There were congruent questions in each instrument in four domains (Scope & Purpose, Stakeholder involvement, Underlying evidence/Rigour, Clarity). The iCAHE and AGREE II scores were moderate to strongly correlated for the six guidelines. There was generally good agreement between testers for iCAHE scores, irrespective of their experience. The iCAHE instrument was preferred by all testers, and took significantly less time to administer than the AGREE II instrument. However, the use of only three testers and six guidelines compromised study power, rendering this research as pilot investigations of the psychometric properties of the iCAHE instrument. CONCLUSION: The iCAHE Guideline Quality Checklist has promising psychometric properties and clinical utility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reliability of the PEDro Scale for Rating Quality of Randomized Controlled Trials
Maher, C. G., Sherrington, C., Herbert, R. D., Moseley, A. M., & Elkins, M. (2003) · Physical Therapy · doi:10.1093/ptj/83.8.713 · Cited by 4,808
AGREE II: advancing guideline development, reporting and evaluation in health care
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., et al. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.090449 · Cited by 3,790
Making psychological theory useful for implementing evidence based practice: a consensus approach
Michie, S. (2005) · Quality and Safety in Health Care · doi:10.1136/qshc.2004.011155 · Cited by 3,440
(2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.18 · Cited by 1,446
Guidelines International Network: Toward International Standards for Clinical Practice Guidelines
Qaseem, A., Forland, F., Macbeth, F., Ollenschläger, G., Phillips, S., van der Wees, P., & for the Board of Trustees of the Guidelines International Network* (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-7-201204030-00009 · Cited by 902
Cited by
A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement
Chen, Y., Yang, K., Marušić, A., Qaseem, A., Meerpohl, J. J., Flottorp, S., Akl, E. A., Schünemann, H. J., Chan, E. S. Y., Falck-Ytter, Y., et al. (2017) · Annals of Internal Medicine · doi:10.7326/m16-1565 · Cited by 665
Guide to clinical practice guidelines: the current state of play
Kredo, T., Bernhardsson, S., Machingaidze, S., Young, T., Louw, Q., Ochodo, E., & Grimmer, K. (2016) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzv115 · Cited by 444
Self-citationRosella, L., Bowman, C., Pach, B., Morgan, S., Fitzpatrick, T., & Goel, V. (2016) · Public Health · doi:10.1016/j.puhe.2015.10.027 · Cited by 90
James, T., Mukadam, N., Sommerlad, A., Pour, H. R., Knowles, M., Azocar, I., & Livingston, G. (2022) · PLOS Medicine · doi:10.1371/journal.pmed.1003860 · Cited by 23
“We used the International Centre for Allied Health Evidence (iCAHE) Guideline Quality Checklist [19] to appraise the quality of included guidelines.”
BackgroundMethodMill, D., Page, A., Johnson, J., Lee, K., Salter, S. M., Seubert, L., Clifford, R., & D’Lima, D. (2022) · BMC Health Services Research · doi:10.1186/s12913-021-07358-4 · Cited by 9
Metadata
- Authors
- Grimmer, K., Dizon, J. M., Milanese, S., King, E., Beaton, K., Thorpe, O., Lizarondo, L., Luker, J., Machotka, Z., & Kumar, S.
- Venue
- BMC Medical Research Methodology
- Published
- 2014-05-10
- Volume
- 14
- Issue
- 1
- Pages
- 63-63
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W1980285844
- Cited by
- 79
- References
- 5
- FWCI
- 5.51
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Efficient clinical evaluation of guideline quality: development and testing of a new tool. https://app.cochat.ai/papers/10.1186/1471-2288-14-63
- The paper
- Grimmer, K., Dizon, J. M., Milanese, S., King, E., Beaton, K., Thorpe, O., Lizarondo, L., Luker, J., Machotka, Z., & Kumar, S. (2014). Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-14-63
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.