The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies
Abstract
BACKGROUND: Despite the increasing number of manuals on how to develop clinical practice guidelines (CPGs) there remain concerns about their quality. The aim of this study was to review the quality of CPGs across a wide range of healthcare topics published since 1980. METHODS: The authors conducted a literature search in MEDLINE to identify publications assessing the quality of CPGs with the Appraisal of Guidelines, Research and Evaluation (AGREE) instrument. For the included guidelines in each study, the authors gathered data about the year of publication, institution, country, healthcare topic, AGREE score per domain and overall assessment. RESULTS: In total, 42 reviews were selected, including a total of 626 guidelines, published between 1980 and 2007, with a median of 25 CPGs. The mean scores were acceptable for the domain 'Scope and purpose' (64%; 95% CI 61.9 to 66.4) and 'Clarity and presentation' (60%; 95% CI 57.9 to 61.9), moderate for domain 'Rigour of development' (43%; 95% CI 41.0 to 45.2), and low for the other domains ('Stakeholder involvement' 35%; 95% CI 33.9 to 37.5, 'Editorial independence' 30%; 95% CI 27.9 to 32.3, and 'Applicability' 22%; 95% CI 20.4 to 23.9). From those guidelines that included an overall assessment, 62% (168/270) were recommended or recommended with provisos. There was a significant improvement over time for all domains, except for 'Editorial independence.' CONCLUSIONS: This review shows that despite some increase in quality of CPGs over time, the quality scores as measured with the AGREE Instrument have remained moderate to low over the last two decades. This finding urges guideline developers to continue improving the quality of their products. International collaboration could help increasing the efficiency of the process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
(2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.18 · Cited by 1,446
Clinical practice guidelines : directions for a new program
Field, M. J., & Lohr, K. N. (1990) · Cited by 1,415
Scientific evidence underlying the ACC/AHA clinical practice guidelines.
Tricoci, P. (2009) · JAMA · doi:10.1001/jama.2009.205 · Cited by 908
Shaneyfelt, T. M., Mayo-Smith, M. F., & Rothwangl, J. (1999) · JAMA · doi:10.1001/jama.281.20.1900 · Cited by 844
Zhang, W., Moskowitz, R. W., Nuki, G., Abramson, S., Altman, R. D., Arden, N., Bierma-Zeinstra, S., Brandt, K. D., Croft, P., Doherty, M., et al. (2007) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2007.06.014 · Cited by 796
Practice guidelines developed by specialty societies: the need for a critical appraisal
Grilli, R., Magrini, N., Penna, A., Mura, G., & Liberati, A. (2000) · The Lancet · doi:10.1016/s0140-6736(99)02171-6 · Cited by 664
Development of the AGREE II, part 1: performance, usefulness and areas for improvement
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Hanna, S. E., Makarski, J., et al. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.091714 · Cited by 499
Self-citationBrouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Hanna, S. E., Makarski, J., et al. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.091716 · Cited by 495
Self-citationShiffman, R. N., Shekelle, P., Overhage, J. M., Slutsky, J., Grimshaw, J., & Deshpande, A. M. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-6-200309160-00013 · Cited by 451
Adaptation of clinical guidelines: literature review and proposition for a framework and procedure
Fervers, B., Burgers, J. S., Haugh, M. C., Latreille, J., Mlika-Cabanne, N., Paquet, L., Coulombe, M., Poirier, M., & Burnand, B. (2006) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzi108 · Cited by 329
Self-citationVlayen, J., Aertgeerts, B., Hannes, K., Sermeus, W., & Ramaekers, D. (2005) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzi027 · Cited by 313
Reassessment of Clinical Practice Guidelines: Go Gently Into That Good Night
Shaneyfelt, T. M., & Centor, R. M. (2009) · JAMA · doi:10.1001/jama.2009.225 · Cited by 294
Improving the Use of Research Evidence in Guideline Development
Oxman, A. D., Schünemann, H. J., Fretheim, A., Boyd, E. A., Bero, L. A., & Edejer, T. T.-T. (2007) · BIBSYS Brage (BIBSYS (Norway)) · Cited by 261
Schizophrenia practice guidelines: International survey and comparison
Gaebel, W., Weinmann, S., Sartorius, N., Rutz, W., & McIntyre, J. S. (2005) · British Journal of Psychiatry · doi:10.1192/bjp.187.3.248 · Cited by 227
Burgers, J. S., Cluzeau, F. A., Hanna, S. E., Hunt, C., & Grol, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646230300014x · Cited by 185
Self-citationQuality of Primary Care Guidelines for Acute Low Back Pain
van Tulder, M. W., Tuut, M. K., Pennick, V., Bombardier, C., & Assendelft, W. J. (2004) · Spine · doi:10.1097/01.brs.0000137056.64166.51 · Cited by 156
Transparent Development of the WHO Rapid Advice Guidelines
Schünemann, H. J., Hill, S. R., Kakad, M., Vist, G. E., Bellamy, R., Stockman, L. J., Wisløff, T. F., Mar, C. D., Hayden, F. G., Uyeki, T. M., et al. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040119 · Cited by 149
Improving the use of research evidence in guideline development: introduction
SURE, Oxman, A. D., Fretheim, A., & Schünemann, H. J. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-12 · Cited by 145
Watine, J., Friedberg, B., Nagy, E., Ónody, R., Oosterhuis, W. P., Bunting, P. S., Charet, J.-C., & Horvath, A. R. (2006) · Clinical Chemistry · doi:10.1373/clinchem.2005.056952 · Cited by 120
Schünemann, H. J., Osborne, M. L., Moss, J., Manthous, C. A., Wagner, G. R., Sicilian, L., Ohar, J. A., McDermott, S., Lucas, L., & Jaeschke, R. (2009) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200901-0126st · Cited by 115
Clinical experts or methodologists to write clinical guidelines?
Hirsh, J., & Guyatt, G. H. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60787-x · Cited by 107
Vitry, A. I., & Zhang, Y. (2008) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb02079.x · Cited by 96
Poitras, S., Avouac, J., Rossignol, M., Avouac, B. P., Cedraschi, C., Nordin, M. C., Rousseaux, C., Rozenberg, S., Savarieau, B., Thoumie, P., et al. (2007) · Arthritis Research & Therapy · doi:10.1186/ar2339 · Cited by 86
Assessment of the Scope and Quality of Clinical Practice Guidelines in Lung Cancer*
Harpole, L. H., Kelley, M. J., Schreiber, G., Toloza, E. M., Kolimaga, J. T., & McCrory, D. C. (2003) · CHEST Journal · doi:10.1378/chest.123.1_suppl.7s · Cited by 82
A vision statement on guideline development for respiratory disease: the example of COPD
Schünemann, H. J., Woodhead, M. A., Anzueto, A., Buist, S. A., MacNee, W., Rabe, K. F., & Heffner, J. E. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61347-1 · Cited by 81
Nuckols, T. K., Lim, Y.-W., Wynn, B. O., Mattke, S., MacLean, C. H., Harber, P. I., Brook, R. H., Wallace, P., Garland, R. H., & Asch, S. M. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0440-9 · Cited by 79
Manchikanti, L., Singh, V., Helm, S., Trescot, A. M., & Hirsch, J. A. (2008) · Pain Physician · doi:10.36076/ppj.2008/11/291 · Cited by 78
Quality of Evidence-Based Pediatric Guidelines
Boluyt, N., Lincke, C. R., & Offringa, M. (2005) · PEDIATRICS · doi:10.1542/peds.2004-0575 · Cited by 74
Improving the use of research evidence in guideline development: 16. Evaluation
Oxman, A. D., Schünemann, H. J., & Fretheim, A. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-28 · Cited by 73
Misso, M. L., Pitt, V. J., Jones, K., Barnes, H., Piterman, L., & Green, S. (2008) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb02086.x · Cited by 67
Major depressive disorder in the general hospital: adaptation of clinical practice guidelines
Voellinger, R., Berney, A., Baumann, P., Annoni, J., Bryois, C., Buclin, T., Büla, C., Camus, V., Christin, L., Cornuz, J., et al. (2003) · General Hospital Psychiatry · doi:10.1016/s0163-8343(03)00009-4 · Cited by 42
Lopez-Olivo, M. A., Kallen, M. A., Ortiz, Z., Skidmore, B., & Suarez-Almazor, M. E. (2008) · Arthritis Care & Research · doi:10.1002/art.24207 · Cited by 40
Delgado‐Noguera, M., Tort, S., Bonfill, X., Gich, I. J., & Alonso‐Coello, P. (2008) · European Journal of Pediatrics · doi:10.1007/s00431-008-0836-5 · Cited by 39
Self-citationAppleyard, T., Mann, C., & Khan, K. (2006) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2006.00937.x · Cited by 34
Subfertility guidelines in Europe: the quantity and quality of intrauterine insemination guidelines
Haagen, E. C., Hermens, R., Nelen, W. L. D. M., Braat, D. D. M., Grol, R. P. T. M., & Kremer, J. A. M. (2006) · Human Reproduction · doi:10.1093/humrep/del100 · Cited by 30
Nagy, E., Watine, J., Bunting, P. S., Ónody, R., Oosterhuis, W. P., Rogić, D., Sandberg, S., Boda, K., & Horvath, A. R. (2008) · Clinical Chemistry · doi:10.1373/clinchem.2008.109082 · Cited by 27
Larsen, T., Kumar, S., Grimmer, K. A., Potter, A., Farquharson, T., & Sharpe, P. (2006) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2006.07.008 · Cited by 22
Watine, J., & Bunting, P. S. (2008) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2007.12.020 · Cited by 21
Clinician Assessment of Guidelines that Support Common Dental Procedures
Faggion, C. M. (2008) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2007.10.001 · Cited by 21
Analysis of the quality of clinical practice guidelines on established ischemic stroke
Puerto, M. A. N., Gutiérrez‐Ibarluzea, I., Ruiz, O., Álvarez, F. M., Herreros, R. G., Pintiado, R. E., Dominguez, A. R., & León, I. M. (2008) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462308080446 · Cited by 21
Guías de práctica clínica en atención primaria: Una evaluación crítica
Tomás Pantoja, C., Strain, H., & Valenzuela, L. (2007) · Revista médica de Chile · doi:10.4067/s0034-98872007001000008 · Cited by 20
Devroey, D., Vantomme, K., Betz, W., Vandevoorde, J., & Kartounian, J. (2004) · Cardiology · doi:10.1159/000077906 · Cited by 15
Baird, A. S., Olarinde, O., & Talbot, M. D. (2007) · Sexual Health · doi:10.1071/sh07048 · Cited by 13
Treatment of unipolar psychotic depression: The use of evidence in practice guidelines
Wijkstra, J., Schubart, C. D., & Nolen, W. A. (2008) · The World Journal of Biological Psychiatry · doi:10.1080/15622970701599052 · Cited by 12
Wennekes, L., Hermens, R., van Heumen, K., Runde, V., Schoelen, H., Wollersheim, H. C. H., Grol, R. P. T. M., Mulder, P. H. M. D., & Ottevanger, P. B. (2008) · The Breast · doi:10.1016/j.breast.2008.01.008 · Cited by 11
Cited by
Alonso-Coello, P., Schünemann, H. J., Moberg, J., Brignardello-Petersen, R., Akl, E. A., Davoli, M., Treweek, S., Mustafa, R. A., Rada, G., Rosenbaum, S., et al. (2016) · BMJ · doi:10.1136/bmj.i2016 · Cited by 1,317
Self-citationWong, J. J., Côté, P., Sutton, D. A., Randhawa, K., Yu, H., Varatharajan, S., Goldgrub, R., Nordin, M., Gross, D. P., Shearer, H. M., et al. (2017) · European Journal of Pain · doi:10.1002/ejp.931 · Cited by 414
Murad, M. H. (2017) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2017.01.001 · Cited by 386
Nagler, E. V., Vanmassenhove, J., van der Veer, S. N., Nistor, I., Van Biesen, W., Webster, A. C., & Vanholder, R. (2014) · BMC Medicine · doi:10.1186/s12916-014-0231-1 · Cited by 309
Gibbs, A. J., Gray, B., Wallis, J. A., Taylor, N. F., Kemp, J. L., Hunter, D. J., & Barton, C. J. (2023) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2023.05.015 · Cited by 277
Metadata
- Authors
- Alonso-Coello, P., Irfan, A., Sola, I., Gich, I., Delgado-Noguera, M., Rigau, D., Tort, S., Bonfill, X., Burgers, J., & Schunemann, H.
- Venue
- BMJ Quality & Safety
- Publisher
- BMJ
- Published
- 2010-12-01
- Volume
- 19
- Issue
- 6
- Pages
- e58-e58
- ISSN
- 1475-3898 (ISSN-L)
- 2044-5415
- 2044-5423
- OpenAlex
- W2140838980
- Cited by
- 472
- References
- 46
- FWCI
- 2.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.