Bridging the Guideline Implementation Gap: A Systematic, Document-Centered Approach to Guideline Implementation
Open accessAbstract
OBJECTIVE: A gap exists between the information contained in published clinical practice guidelines and the knowledge and information that are necessary to implement them. This work describes a process to systematize and make explicit the translation of document-based knowledge into workflow-integrated clinical decision support systems. DESIGN: This approach uses the Guideline Elements Model (GEM) to represent the guideline knowledge. Implementation requires a number of steps to translate the knowledge contained in guideline text into a computable format and to integrate the information into clinical workflow. The steps include: (1) selection of a guideline and specific recommendations for implementation, (2) markup of the guideline text, (3) atomization, (4) deabstraction and (5) disambiguation of recommendation concepts, (6) verification of rule set completeness, (7) addition of explanations, (8) building executable statements, (9) specification of origins of decision variables and insertions of recommended actions, (10) definition of action types and selection of associated beneficial services, (11) choice of interface components, and (12) creation of requirement specification. RESULTS: The authors illustrate these component processes using examples drawn from recent experience translating recommendations from the National Heart, Lung, and Blood Institute's guideline on management of chronic asthma into a workflow-integrated decision support system that operates within the Logician electronic health record system. CONCLUSION: Using the guideline document as a knowledge source promotes authentic translation of domain knowledge and reduces the overall complexity of the implementation task. From this framework, we believe that a better understanding of activities involved in guideline implementation will emerge.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Guideline documents as a knowledge source reduce the overall complexity of the implementation task.
“Guideline documents as a knowledge source promote authentic translation of domain knowledge and reduce the overall complexity of the implementation task [123].”
From Health Care Provider Adoption of eHealth: Systematic Literature Review (2013) · cites this paper as [123]
Guideline documents as a knowledge source promote authentic translation of domain knowledge.
“Guideline documents as a knowledge source promote authentic translation of domain knowledge and reduce the overall complexity of the implementation task [123].”
From Health Care Provider Adoption of eHealth: Systematic Literature Review (2013) · cites this paper as [123]
Cites
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,754 · see citing works
Why Don't Physicians Follow Clinical Practice Guidelines?: A Framework for Improvement
Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H., Abboud, P.-A. C., & Rubin, H. R. (1999) · JAMA · doi:10.1001/jama.282.15.1458 · Cited by 6,677 · see citing works
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,592 · see citing works
Oxman, A. D., Thomson, M., Davis, D. A., & Haynes, R. B. (1995) · PubMed Central · Cited by 1,847 · see citing works
Guidelines for Clinical Practice
Institute of Medicine (1992) · National Academies Press eBooks · doi:10.17226/1863 · Cited by 916 · see citing works
Grol, R., Dalhuijsen, J., Thomas, S., Veld, C. I. ‘., Rutten, G., & Mokkink, H. (1998) · BMJ · doi:10.1136/bmj.317.7162.858 · Cited by 877 · see citing works
Are Guidelines Following Guidelines?
Shaneyfelt, T. M., Mayo‐Smith, M. F., & Rothwangl, J. (1999) · JAMA · doi:10.1001/jama.281.20.1900 · Cited by 846 · see citing works
Patient care information systems and health care work: a sociotechnical approach
Berg, M. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00011-8 · Cited by 786 · see citing works
Design Patterns for Object-Oriented Software Development
Pree, W., & Sikora, H. (1994) · Cited by 562 · see citing works
Comparing Computer-interpretable Guideline Models: A Case-study Approach
Mor Peleg, B., Tu, S. W., Bury, J. P., Ciccarese, P., Fox, J., Greenes, R. A., Hall, R., Johnson, P. D., Jones, N., Kumar, A., et al. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1135 · Cited by 551 · see citing works
Shiffman, R. N., Shekelle, P., Overhage, J. M., Slutsky, J. R., Grimshaw, J., & Deshpande, A. M. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-6-200309160-00013 · Cited by 451 · see citing works
Self-citationThe GuideLine Interchange Format: A Model for Representing Guidelines
Ohno‐Machado, L., Gennari, J. H., Murphy, S. N., et al. (1998) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1998.0050357 · Cited by 378 · see citing works
An analysis of physician attitudes regarding computer-based clinical consultation systems
Teach, R. L., & Shortliffe, E. H. (1981) · Computers and Biomedical Research · doi:10.1016/0010-4809(81)90012-4 · Cited by 330 · see citing works
GEM: A Proposal for a More Comprehensive Guideline Document Model Using XML
Shiffman, R. N., Karras, B. T., Agrawal, A., Chen, R., Marenco, L. N., & Nath, S. (2000) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2000.0070488 · Cited by 277 · see citing works
Self-citationComputerizing Guidelines to Improve Care and Patient Outcomes: The Example of Heart Failure
Tierney, W. M., Overhage, J. M., Takesue, B. Y., Harris, L. E., Murray, M. D., Vargo, D. L., & McDonald, C. J. (1995) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1995.96073834 · Cited by 214 · see citing works
Optimal Methods for Guideline Implementation
Gross, P. A., Greenfield, S., Cretin, S., Ferguson, J., Grimshaw, J., Grol, R. P. T. M., Klazinga, N., Lorenz, W., Meyer, G. S., Riccobono, C. A., et al. (2001) · Medical Care · doi:10.1097/00005650-200108002-00006 · Cited by 168 · see citing works
Representing Clinical Guidelines in GLIF: Individual and Collaborative Expertise
Patel, V. L., Allen, V. G., Arocha, J. F., & Shortliffe, E. H. (1998) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1998.0050467 · Cited by 119 · see citing works
Changing professional practice: Theory and practice of clinical guidelines implementation
Thorsen, T., & Mākelä, M. (1999) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 115 · see citing works
Representation of Clinical Practice Guidelines in Conventional and Augmented Decision Tables
Shiffman, R. N. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040382 · Cited by 115 · see citing works
Self-citationAutomating Complex Guidelines for Chronic Disease: Lessons Learned
Maviglia, S. M., Zielstorff, R. D., Paterno, M. D., Teich, J. M., Bates, D. W., & Kuperman, G. J. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1181 · Cited by 109 · see citing works
A virtual medical record for guideline-based decision support.
Johnson, P. D., Tu, S. W., Musen, M. A., et al. (2001) · PubMed · Cited by 100 · see citing works
Improving Clinical Guidelines with Logic and Decision-table Techniques
Shiffman, R. N., & Greenes, R. A. (1994) · Medical Decision Making · doi:10.1177/0272989x9401400306 · Cited by 76 · see citing works
Self-citationModeling guidelines for integration into clinical workflow.
Tu, S. W., Musen, M. A., Shankar, R., et al. (2004) · PubMed · Cited by 68 · see citing works
A Design Model for Computer-based Guideline Implementation Based on Information Management Services
Shiffman, R. N., Brandt, C., Liaw, Y., & Corb, G. J. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.0060099 · Cited by 68 · see citing works
Self-citationCOMPUTERIZED DECISION SUPPORT SYSTEMS IN PRIMARY CARE*
Elson, R. B., & Connelly, D. P. (1995) · Primary Care Clinics in Office Practice · doi:10.1016/s0095-4543(21)00523-6 · Cited by 51 · see citing works
A model for adapting clinical guidelines for electronic implementation in primary care.
Lobach, D. F. (1995) · PubMed · Cited by 36 · see citing works
Shiffman, R. N. (1994) · PubMed · Cited by 33 · see citing works
Self-citationEssaihi, A., Michel, G., & Shiffman, R. N. (2003) · PubMed · Cited by 27 · see citing works
Self-citationElson, R. B. (1997) · Disease Management & Health Outcomes · doi:10.2165/00115677-199701020-00001 · Cited by 25 · see citing works
Support for guideline development through error classification and constraint checking.
Peleg, M., Patel, V. L., Snow, V. T., et al. (2002) · PubMed · Cited by 24 · see citing works
Gersende Georg, A., Brigitte Séroussi, A., & Jacques Bouaud, A. (2003) · PubMed · Cited by 21 · see citing works
Using GEM-encoded guidelines to generate medical logic modules.
Agrawal, A., & Shiffman, R. N. (2001) · PubMed · Cited by 20 · see citing works
Self-citationAn implementation framework for GEM encoded guidelines.
Gershkovich, P., & Shiffman, R. N. (2001) · PubMed · Cited by 14 · see citing works
Self-citationEvaluation of guideline quality using GEM-Q.
Agrawal, A., & Shiffman, R. N. (2001) · PubMed · Cited by 11 · see citing works
Self-citationMcArthur, R. P. (1990) · Medical Entomology and Zoology · Cited by 3 · see citing works
Shiffman, R. N., Michel, G., & Essaihi, A. (2003) · HIC 2003 RACGP12CC [combined conference]: Proceedings · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Health Care Provider Adoption of eHealth: Systematic Literature Review
Li, J., Talaei-Khoei, A., Seale, H., Ray, P., & MacIntyre, C. R. (2013) · interactive Journal of Medical Research · doi:10.2196/ijmr.2468 · Cited by 250 · see citing works
Optimizing the language and format of guidelines to improve guideline uptake
Gupta, S., Rai, N., Bhattacharrya, O., Cheng, A. Y. Y., Connelly, K. A., Boulet, L.-P., Kaplan, A., Brouwers, M. C., & Kastner, M. (2016) · Canadian Medical Association Journal · doi:10.1503/cmaj.151102 · Cited by 82 · see citing works
- Authors
- Shiffman, R. N., Michel, G., Essaihi, A., & Thornquist, E.
- Affiliations
- Yale New Haven Hospital (US)
- Corresponding author
- Shiffman, Richard N.
- ORCID
- Michel, George
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-09
- Volume
- 11
- Issue
- 5
- Pages
- 418-426
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC516249
- OpenAlex
- W1498969322
- Cited by
- 204
- References
- 36
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 5 · 2023: 6 · 2022: 7 · 2021: 7 · 2020: 9 · 2019: 5 · 2018: 7 · 2017: 4 · 2016: 3 · 2015: 3 · 2014: 11 · 2013: 11 · 2012: 11
- FWCI
- 32.13
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (5r29lm005552-05)
- U.S. National Library of Medicine (R01LM007199, 1 R01-LM-07199, T15-LM07065, R29- LM05552)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- guideline implementation, clinical decision support systems, knowledge translation, clinical workflow integration, requirements specification
- MeSH terms
- Asthma, Humans, Programming Languages, Software, Medical Records Systems, Computerized, Practice Guidelines as Topic, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
