The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Background: Clinical practice guidelines are not uniformly successful in influencing clinicians' behaviour toward best practices. Implementability refers to a set of characteristics that predict ease of (and obstacles to) guideline implementation. Our objective is to develop and validate a tool for appraisal of implementability of clinical guidelines. Methods: Indicators of implementability were identified from the literature and used to create items and dimensions of the GuideLine Implementability Appraisal (GLIA). GLIA consists of 31 items, arranged into 10 dimensions. Questions from 9 of the 10 dimensions are applied individually to each recommendation of the guideline. Decidability and Executability are critical dimensions. Other dimensions are Global, Presentation and Formatting, Measurable Outcomes, Apparent Validity, Flexibility, Effect on Process of Care, Novelty/Innovation, and Computability. We conducted a series of validation activities, including validation of the construct of implementability, expert review of content for clarity, relevance, and comprehensiveness, and assessment of construct validity of the instrument. Finally, GLIA was applied to a draft guideline under development by national professional societies. Results: Evidence of content validity and preliminary support for construct validity were obtained. The GLIA proved to be useful in identifying barriers to implementation in the draft guideline and the guideline was revised accordingly. Conclusion: GLIA may be useful to guideline developers who can apply the results to remedy defects in their guidelines. Likewise, guideline implementers may use GLIA to select implementable recommendations and to devise implementation strategies that address identified barriers. By aiding the design and operationalization of highly implementable guidelines, our goal is that application of GLIA may help to improve health outcomes, but further evaluation will be required to support this potential benefit.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Determination and Quantification Of Content Validity
LYNN, M. R. (1986) · Nursing Research · doi:10.1097/00006199-198611000-00017 · Cited by 6,775
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044
ACC/AHA guidelines for the management of patients with acute myocardial infarction
Committee Members, Ryan, T. J., Anderson, J. L., Antman, E. M., Braniff, B. A., Brooks, N. H., Califf, R. M., Hillis, L. D., Hiratzka, L. F., Rapaport, E., et al. (1996) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(96)00392-0 · Cited by 1,850
Hunt, D. L., Haynes, R. B., Hanna, S. E., & Smith, K. (1998) · JAMA · doi:10.1001/jama.280.15.1339 · Cited by 1,669
(2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.18 · Cited by 1,446
(2000) · Age and Ageing · doi:10.1093/oxfordjournals.ageing.a008095 · Cited by 1,382
Changing Physicians' Practices
Greco, P. J., & Eisenberg, J. M. (1993) · New England Journal of Medicine · doi:10.1056/nejm199310213291714 · Cited by 977
Committee on Quality Improvement, S. O. U. T. I. (1999) · Pediatrics · doi:10.1542/peds.103.4.843 · Cited by 948
Guidelines for Clinical Practice: From Development to Use
(1992) · National Academies Press eBooks · doi:10.17226/1863 · Cited by 915
Grol, R., Dalhuijsen, J., Thomas, S., Veld, C. I. ‘., Rutten, G., & Mokkink, H. (1998) · BMJ · doi:10.1136/bmj.317.7162.858 · Cited by 877
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., et al. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305290190 · Cited by 864
Shaneyfelt, T. M., Mayo-Smith, M. F., & Rothwangl, J. (1999) · JAMA · doi:10.1001/jama.281.20.1900 · Cited by 844
Ryan, T. J., Antman, E. M., Brooks, N. H., Califf, R. M., Hillis, L. D., Hiratzka, L. F., Rapaport, E., Riegel, B., Russell, R. O., Smith, E. E., et al. (1999) · Circulation · doi:10.1161/01.cir.100.9.1016 · Cited by 743
American Academy of Family Physicians, American Academy of Otolaryngology-Head and Neck Surgery, & American Academy of Pediatrics Subcommittee on Otitis Media With Effusion (2004) · Pediatrics · doi:10.1542/peds.113.5.1412 · Cited by 475
Shiffman, R. N., Shekelle, P., Overhage, J. M., Slutsky, J., Grimshaw, J., & Deshpande, A. M. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-6-200309160-00013 · Cited by 451
Self-citationGrilli, R., & Lomas, J. (1994) · Medical Care · doi:10.1097/00005650-199403000-00002 · Cited by 392
Evaluating informatics applications – clinical decision support systems literature review
Kaplan, B. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00183-6 · Cited by 350
Shiffman, R. N., Liaw, Y., Brandt, C. A., & Corb, G. J. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.0060104 · Cited by 333
Self-citationTowards evidence-based clinical practice: an international survey of 18 clinical guideline programs
BURGERS, J. S. (2003) · International Journal for Quality in Health Care · doi:10.1093/intqhc/15.1.31 · Cited by 331
CLINICAL GUIDELINE: Diagnosing Syncope: Part 2: Unexplained Syncope
Linzer, M., Yang, E. H., Estes, N. A. M., Wang, P., Vorperian, V. R., & Kapoor, W. N. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-1-199707010-00014 · Cited by 313
Changing clinical behaviour by making guidelines specific
Michie, S., & Johnston, M. (2004) · BMJ · doi:10.1136/bmj.328.7435.343 · Cited by 309
Development and application of a generic methodology to assess the quality of clinical guidelines
Cluzeau, F., Littlejohns, P., Grimshaw, J., Feder, G., & Moran, S. (1999) · International Journal for Quality in Health Care · doi:10.1093/intqhc/11.1.21 · Cited by 271
Lessons from Experienced Guideline Implementers: Attend to Many Factors and Use Multiple Strategies
Solberg, L. I., Brekke, M. L., Fazio, C. J., Fowles, J., Jacobsen, D. N., Kottke, T. E., Mosser, G., O’Connor, P. J., Ohnsorg, K. A., & Rolnick, S. J. (2000) · The Joint Commission Journal on Quality Improvement · doi:10.1016/s1070-3241(00)26013-6 · Cited by 259
Computerizing Guidelines to Improve Care and Patient Outcomes: The Example of Heart Failure
Tierney, W. M., Overhage, J. M., Takesue, B. Y., Harris, L. E., Murray, M. D., Vargo, D. L., & McDonald, C. J. (1995) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1995.96073834 · Cited by 214
Tierney, W. M., Overhage, J. M., Murray, M. D., Harris, L. E., Zhou, X.-H., Eckert, G. J., Smith, F. E., Nienaber, N., McDonald, C. J., & Wolinsky, F. D. (2003) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2003.30635.x · Cited by 199
Shekelle, P., Kravitz, R. L., Beart, J., Marger, M., Wang, M., & Lee, M. L. (2000) · PubMed · Cited by 155
Ollenschlager, G. (2004) · Quality and Safety in Health Care · doi:10.1136/qhc.13.6.455 · Cited by 137
A COMPARISON OF CLINICAL PRACTICE GUIDELINE APPRAISAL INSTRUMENTS
Graham, I. D., Calder, L. A., Hébert, P. C., Carter, A. O., & Tetroe, J. M. (2000) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300103095 · Cited by 131
Representation of Clinical Practice Guidelines in Conventional and Augmented Decision Tables
Shiffman, R. N. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040382 · Cited by 114
Self-citationChanging professional practice: Theory and practice of clinical guidelines implementation
Thorsen, T., & Mākelä, M. (1999) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 113
Status of practice guidelines in the United States: CDC guidelines as an example
Larson, E. (2003) · Preventive Medicine · doi:10.1016/s0091-7435(03)00014-8 · Cited by 63
Katz, D. A. (1999) · PubMed Central · Cited by 61
Cited by
Clinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Clinical Practice Guideline: Otitis Media with Effusion (Update)
Rosenfeld, R. M., Shin, J. J., Schwartz, S. R., Coggins, R., Gagnon, L., Hackell, J. M., Hoelting, D., Hunter, L. L., Kummer, A. W., Payne, S. C., et al. (2016) · Otolaryngology–Head and Neck Surgery · doi:10.1177/0194599815623467 · Cited by 2,006
Clinical Practice Guideline (Update): Adult Sinusitis
Rosenfeld, R. M., Piccirillo, J. F., Chandrasekhar, S. S., Brook, I., Ashok Kumar, K., Kramper, M., Orlandi, R. R., Palmer, J. N., Patel, Z. M., Peters, A., et al. (2015) · Otolaryngology–Head and Neck Surgery · doi:10.1177/0194599815572097 · Cited by 1,159
Clinical Practice Guideline: Benign Paroxysmal Positional Vertigo
Bhattacharyya, N., Baugh, R. F., Orvidas, L., Barrs, D., Bronston, L. J., Cass, S., Chalian, A. A., Desmond, A. L., Earll, J. M., Fife, T. D., et al. (2008) · Otolaryngology–Head and Neck Surgery · doi:10.1016/j.otohns.2008.08.022 · Cited by 1,061
Clinical practice guideline on diagnosis and treatment of hyponatraemia
Spasovski, G., Vanholder, R., Allolio, B., Annane, D., Ball, S., Bichet, D., Decaux, G., Fenske, W., Hoorn, E., Ichai, C., et al. (2014) · Intensive Care Medicine · doi:10.1007/s00134-014-3210-2 · Cited by 1,020
Metadata
- Authors
- Shiffman, R. N., Dixon, J., Brandt, C., Essaihi, A., Hsiao, A., Michel, G., & O'Connell, R.
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2005-07-27
- Volume
- 5
- Issue
- 1
- Pages
- 23-23
- ISSN
- 1472-6947 (ISSN-L)
- OpenAlex
- W1602739543
- Cited by
- 389
- References
- 33
- FWCI
- 6.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.