The state of the art in clinical knowledge management: an inventory of tools and techniques
Open accessAbstract
Purpose To explore the need for, and use of, high-quality, collaborative, clinical knowledge management (CKM) tools and techniques to manage clinical decision support (CDS) content.Methods In order to better understand the current state of the art in CKM, we developed a survey of potential CKM tools and techniques. We conducted an exploratory study by querying a convenience sample of respondents about their use of specific practices in CKM.Results The following tools and techniques should be priorities in organizations interested in developing successful computer-based provider order entry (CPOE) and CDS implementations: (1) a multidisciplinary team responsible for creating and maintaining the clinical content; (2) an external organizational repository of clinical content with web-based viewer that allows anyone in the organization to review it; (3) an online, collaborative, interactive, Internet-based tool to facilitate content development; (4) an enterprise-wide tool to maintain the controlled clinical terminology concepts. Even organizations that have been successfully using computer-based provider order entry with advanced clinical decision support features for well over 15 years are not using all of the CKM tools or practices that we identified.Conclusions If we are to further stimulate progress in the area of clinical decision support, we must continue to develop and refine our understanding and use of advanced CKM capabilities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A key challenge with clinical decision support is knowledge management as guidelines and best practices change over time, as well as governance.
“A key challenge with clinical decision support is knowledge management as guidelines and best practices change over time,40–42 as well as governance.43–45”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Institute of Medicine (2000) · National Academies Press eBooks · doi:10.17226/9728 · Cited by 14,174 · see citing works
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,754 · see citing works
Alavi, M., & Leidner, D. E. (2001) · MIS Quarterly · doi:10.2307/3250961 · Cited by 10,176 · see citing works
To Err is Human: Building a Safer Health System
Hendee, W. R. (2001) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(01)70072-3 · Cited by 10,122 · see citing works
Crossing the Quality Chasm: A New Health System for the 21st Century
Committee on Quality of Health Care in America (2002) · Journal for Healthcare Quality · doi:10.1111/j.1945-1474.2002.tb00463.x · Cited by 8,508 · see citing works
Wall, A. (2000) · British Journal of Healthcare Management · doi:10.12968/bjhc.2000.6.9.19311 · Cited by 5,909 · see citing works
Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S. C., & Shekelle, P. G. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-10-200605160-00125 · Cited by 3,159 · cited 1× · see citing works
“Four of the six have been recognized for their excellence in implementation and use of advanced clinical information systems including use of clinical decision support features and functions [11,12].”
BackgroundProtocol-Based Computer Reminders, the Quality of Care and the Non-Perfectability of Man
McDonald, C. J. (1976) · New England Journal of Medicine · doi:10.1056/nejm197612092952405 · Cited by 804 · cited 2× · see citing works
“…that the organization felt was critical for their success [22], and many sites had a significant number of similarly functioning clinical decision support interventions developed within their systems before any of these relatively new standards were developed (see for example, McDonald, 1976 [23].”
Grand challenges in clinical decision support
Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., Campbell, E., & Bates, D. W. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.09.003 · Cited by 582 · cited 1× · see citing works
“In a recent article, we identified “disseminate best practices in Clinical Decision Support (CDS) design, development, and implementation” as one of the “grand challenges” in CDS [7].”
Self-citationElectronic Health Record Use and the Quality of Ambulatory Care in the United States
Linder, J. A., Ma, J., Bates, D. W., Middleton, B., & Stafford, R. S. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.13.1400 · Cited by 438 · cited 1× · see citing works
“Recent reports have reiterated the fact that, for these clinical information systems to deliver their expected improvements in quality, they must be equipped with state of the art, real-time, point-of-care, clinical decision support (CDS) features [3].”
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · cited 1× · see citing works
“Some sites reported making substantial modifications to the clinical content, for example, selecting a small subset of the most clinically significant drug-drug interactions for use in alerting [20].”
BackgroundBarriers and facilitators to the use of knowledge management systems
Damodaran, L., & Olphert, W. (2000) · Behaviour and Information Technology · doi:10.1080/014492900750052660 · Cited by 305 · cited 1× · see citing works
“the human, cultural and organizational issues) relevant to knowledge management systems” are critical to their widespread use and ultimate success [31]).”
BackgroundImproving Outcomes with Clinical Decision Support : An Implementer's Guide
Osheroff, J. A., Teich, J. M., Levick, D., Saldana, L., Velasco, F. T., Sittig, D. F., Rogers, K. M., & Jenders, R. A. (2012) · Cited by 297 · see citing works
Self-citationA Consensus Statement on Considerations for a Successful CPOE Implementation
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1204 · Cited by 239 · cited 3× · see citing works
“[48] J.”
BackgroundSelf-citationTowards an integrated approach to knowledge management: ‘hard’, ‘soft’ and ‘abstract’ issues
Hlupić, V., Pouloudi, A., & Rzevski, G. (2002) · Knowledge and Process Management · doi:10.1002/kpm.134 · Cited by 187 · see citing works
Sirajuddin, A. M., Osheroff, J. A., Sittig, D. F., Chuo, J., Velasco, F. T., & Collins, D. A. (2009) · PubMed · Cited by 131 · see citing works
Self-citationSupporting infobuttons with terminological knowledge.
Cimino, J. J., Elhanan, G., & Zeng, Q. T. (1997) · PubMed · Cited by 110 · see citing works
Creating and sharing clinical decision support content with Web 2.0: Issues and examples
Wright, A. T., Bates, D. W., Blackford Middleton, A., Hongsermeier, T., Kashyap, V., Thomas, S. M., & Sittig, D. F. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.09.003 · Cited by 106 · see citing works
Self-citationGoldstein, M. K., Coleman, R. W., Tu, S. W., Shankar, R., O’Connor, M. J., Musen, M. A., Martins, S. B., Lavori, P. W., Shlipak, M. G., Oddone, E. Ž., et al. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1534 · Cited by 104 · cited 2× · see citing works
“, alert, order set, and patient education material) have an individual responsible for monitoring the underlying clinical knowledge and maintaining the CDS intervention [48]? 7: Can your system send automatic reminders that content review date is approaching to individuals responsible for maintaining the content [49]?…”
KnowledgeLink: Impact of Context-Sensitive Information Retrieval on Clinicians' Information Needs
Maviglia, S. M., Yoon, C. S., Bates, D. W., & Kuperman, G. J. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1861 · Cited by 99 · cited 1× · see citing works
“Furthermore, several organizations have implemented “infobutton-like” functionality by hard-coding uniform resource locators (URLs) into their system to provide users with key patient-specific or contextsensitive links to reference information directly from the screen that presents the patient’s current medications (t…”
BackgroundThe Medical Gopher--a microcomputer system to help find, organize and decide about patient data.
McDonald, C. J., & Tierney, W. M. (1986) · PubMed · Cited by 90 · see citing works
Kuperman, G. J., Reichley, R. M., & Bailey, T. C. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2055 · Cited by 78 · see citing works
Desiderata for a clinical terminology server.
Chute, C. G., Elkin, P. L., Sherertz, D. D., & Tuttle, M. S. (1999) · PubMed Central · Cited by 68 · see citing works
A framework and model for evaluating clinical decision support architectures
Wright, A. T., & Sittig, D. F. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.03.009 · Cited by 68 · cited 1× · see citing works
“There were many reasons for this including: the CIS vendors did not support these standards; the standards were lacking in one or more key features that the organization felt was critical for their success [22], and many sites had a significant number of similarly functioning clinical decision support interventions de…”
Self-citationComputer-Interpretable Guideline formalisms.
de Clercq, P. A., Kaiser, K. M. A., & Hasman, A. (2008) · PubMed · Cited by 63 · see citing works
Terminology Tools: State of the Art and Practical Lessons
Cimino, J. J. (2001) · Methods of Information in Medicine · doi:10.1055/s-0038-1634425 · Cited by 48 · cited 1× · see citing works
“, more than 100 rule-based alerts or over 100 order sets) clinical decision support operations [15,16].”
(2005) · IGI Global eBooks · doi:10.4018/978-1-59140-300-5 · Cited by 45 · cited 1× · see citing works
“A key component of any CDS initiative is how the organization develops, disseminates, maintains, and evaluates its clinical knowledge content, or its Clinical Knowledge Management practices [8].”
BackgroundWright, A. T., & Sittig, D. F. (2006) · PubMed Central · Cited by 43 · see citing works
Self-citationKaiser Permanente's Convergent Medical Terminology.
Dolin, R. H., Mattison, J. E., Cohn, S. P., Campbell, K. E., Wiesenthal, A. M., Hochhalter, B., Laberge, D., Barsoum, R., Shalaby, J., Abilla, A., et al. (2004) · PubMed · Cited by 42 · see citing works
Practical experience with the maintenance and auditing of a large medical ontology
Baorto, D. M., Li, L., & Cimino, J. J. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.03.005 · Cited by 37 · cited 1× · see citing works
“, more than 100 rule-based alerts or over 100 order sets) clinical decision support operations [15,16].”
Guideline Maintenance and Revision
Shiffman, R. N. (1995) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpedi.1995.02170200017002 · Cited by 35 · cited 1× · see citing works
“Moreover, this clinical content must be kept up to date as the existing knowledge changes, new clinical knowledge is gained, and underlying changes take place in the clinical information system [6].”
BackgroundAutomated bibliographic retrieval based on current topics in hepatology: Hepatopix
Powsner, S. M., Riely, C. A., Barwick, K. W., Morrow, J. S., & Miller, P. L. (1989) · Computers and Biomedical Research · doi:10.1016/0010-4809(89)90075-x · Cited by 32 · cited 2× · see citing works
“) to execute its clinical decision support [41–46]? 5: What types of people do you have to help manage your clinical knowledge [47]? 6: Does each piece of clinical content (i.”
BackgroundComputer-interpretable Guideline Formalisms
Paul, D. C., Katharina, K., & Arie, H. (2008) · Studies in health technology and informatics · doi:10.3233/978-1-58603-873-1-22 · Cited by 32 · cited 2× · see citing works
“) to execute its clinical decision support [41–46]? 5: What types of people do you have to help manage your clinical knowledge [47]? 6: Does each piece of clinical content (i.”
BackgroundResponse to a trial of physician-based inpatient order entry.
Teich, J. M., Spurr, C. D., Flammini, S., Schmiz, J. L., Beckley, R. F., Hurley, J. F., Aranow, M., & GLASER, J. P. (1993) · PubMed Central · Cited by 30 · see citing works
Geissbühler, A., & Miller, R. A. (1999) · PubMed · Cited by 29 · see citing works
Automated Online Transition from the Medical Record to the Psychiatric Literature
Powsner, S. M., & Miller, P. L. (1992) · Methods of Information in Medicine · doi:10.1055/s-0038-1634873 · Cited by 27 · cited 2× · see citing works
“) to execute its clinical decision support [41–46]? 5: What types of people do you have to help manage your clinical knowledge [47]? 6: Does each piece of clinical content (i.”
BackgroundJensen, J. (2006) · PubMed · doi:10.6083/m4hd7smv · Cited by 15 · see citing works
A process to maintain the quality of a computerized knowledge base.
Kuperman, G. J., Fiskio, J. M., & Karson, A. S. (1999) · PubMed · Cited by 11 · see citing works
Rocha, R. A., Bradshaw, R. L., Bigelow, S. M., Hanna, T. P., Fiol, G. D., Hulse, N. C., Roemer, L. K., & Wilkinson, S. G. (2006) · PubMed Central · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- [25]Sittig DF Wright A Osheroff JA Middleton B Teich JM Ash JS Campbell E Bates DW Grand challenges in clinical decision support J Biomed Inform Apr 2008 41 2 387 92 18029232 PMC2660274
- [30]Alavi M Leidner DE Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues MIS Quarterly 2001 25 1 107 136
- [33]Chin HL Overhage JM The comprehensive computer-based patient record in Kaiser Permanente Northwest Proc Fourth Ann Davies Award 1998 69 129 McGraw-Hill New York
- [41]Shah NR Seger AC Seger DL Fiskio JM Kuperman GJ Blumenfeld B Recklet EG Bates DW Gandhi TK Improving acceptance of computerized prescribing alerts in ambulatory care J Am Med Inform Assoc Jan-Feb 2006 13 1 5 11 16221941 PMC1380196
- [44]Wright A Sittig DF A framework and model for evaluating clinical decision support architectures J Biomed Inform Dec 2008 41 6 982 90 18462999 PMC2638589
- [49]Osheroff JA Improving Medication Use and Outcomes with Clinical Decision Support A Step-by-Step Guide 2009 Healthcare Information and Management Systems Society PMC3316472 19894486
- [52]Shiffman RN Guideline maintenance and revision. 50 years of the Jones criteria for diagnosis of rheumatic fever Arch Pediatr Adolesc Med Jul 1995 149 7 727 32 7795761
- [53]Sittig DF Wright A Osheroff JA Middleton B Teich JM Ash JS Campbell E Bates DW Grand challenges in clinical decision support J Biomed Inform Apr 2008 41 2 387 92 18029232 PMC2660274
- [55]Dolin RH Mattison JE Cohn S Kaiser Permanente’s Convergent Medical Terminology Medinfo 2004 11 Pt 1 346 50 15360832
Cited by
Sorted by citations
Sittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
“Certain elements of the clinical content, such as that which informs CDS interventions, must be managed on a regular basis.(36)”
BackgroundSelf-citationHorsky, J., Schiff, G. D., Johnston, D., Mercincavage, L. M., Bell, D. S., & Blackford Middleton, A. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.09.002 · Cited by 290 · see citing works
Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A. T., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 276 · see citing works
“…integrated into the clinicians’ workflow at the point of care is one of the main benefits associated with EHRs, and this has been successfully accomplished for Infobuttons [227-229], order sets [230, 231], documentation templates [232-234], data displays/flowsheets, as well as alerts and reminders.”
BackgroundSelf-citationClinical decision support systems
Beeler, P. E., Bates, D. W., & Hug, B. L. (2014) · Swiss Medical Weekly · doi:10.4414/smw.2014.14073 · Cited by 157 · see citing works
Analysis of clinical decision support system malfunctions: a case series and survey
Wright, A., Hickman, T.-T. T., McEvoy, D., Aaron, S., Ai, A., Andersen, J. M., Hussain, S., Ramoni, R., Fiskio, J., Sittig, D. F., et al. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw005 · Cited by 132 · see citing works
Self-citationShahmoradi, L., صفدری, ر., & Jimma, W. (2017) · Ethiopian Journal of Health Sciences · doi:10.4314/ejhs.v27i5.13 · Cited by 123 · see citing works
“Accordingly, four tools were identified (43).”
BackgroundInfluentialHigh-priority drug–drug interactions for use in electronic health records
Phansalkar, S., Desai, A. A., Bell, D., Yoshida, E., Doole, J., Czochanski, M., Middleton, B., & Bates, D. W. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000612 · Cited by 134 · see citing works
Design of decision support interventions for medication prescribing
Horsky, J., Phansalkar, S., Desai, A. A., Bell, D. S., & Blackford Middleton, A. (2013) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2013.02.003 · Cited by 108 · see citing works
Ash, J. S., Sittig, D. F., Guappone, K. P., Dykstra, R. H., Richardson, J., Wright, A., Carpenter, J., McMullen, C., Shapiro, M., Bunce, A., et al. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-6 · Cited by 103 · see citing works
“Before each site visit, we asked a local on-site expert to complete a “site profile,” a checklist of types of CDS and questions about CDS management [31].”
MethodSelf-citationComputerised decision support systems for healthcare professionals: an interpretative review
Cresswell, K. M., Majeed, A., Bates, D. W., & Sheikh, A. (2013) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v20i2.32 · Cited by 101 · see citing works
Wright, A., Sittig, D. F., Ash, J. S., Bates, D. W., Feblowitz, J., Fraser, G., Maviglia, S. M., McMullen, C., Nichol, W. P., Pang, J. E., et al. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002030 · Cited by 92 · see citing works
“This content must be engineered (or purchased) and must also be kept current as clinical knowledge, guidelines, and best practices evolve.(6) The knowledge bases that are developed must reflect both universal best practices and local practices and needs.”
BackgroundSelf-citationA pilot study of distributed knowledge management and clinical decision support in the cloud
Dixon, B. E., Simonaitis, L., Goldberg, H., Paterno, M. D., Schaeffer, M. H., Hongsermeier, T., Wright, A. T., & Blackford Middleton, A. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.03.004 · Cited by 85 · see citing works
Self-citationClinical decision support alert malfunctions: analysis and empirically derived taxonomy
Wright, A. T., Ai, A., Ash, J. S., Wiesen, J. F., Hickman, T.-T. T., Aaron, S., McEvoy, D. S., Borkowsky, S., Dissanayake, P. I., Embí, P. J., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx106 · Cited by 79 · see citing works
Self-citationA distributed clinical decision support system architecture
El–Sappagh, S., & El‐Masri, S. (2013) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2013.03.005 · Cited by 74 · see citing works
“This module can represent the extracted knowledge by using production rules, semantic networks/web, frames, decision tables, decision trees or conceptual graphs (Tanwar et al., 2010a,b; Chakr-aborty, 2008; Priss, 2010; Sittig et al., 2010).”
Fung, K. W., Kapusnik-Uner, J., Cunningham, J., Higby-Baker, S., & Bodenreider, O. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx010 · Cited by 74 · see citing works
“Currently, most health care organizations still rely on a single proprietary KB.(26) Inconsistent evaluation and classification of interactions have been cited as factors contributing to excessive DDI alerts.”
BackgroundInfluentialAckerhans, S., Huynh, T. V., Kaiser, C., & Schultz, C. (2024) · Implementation Science · doi:10.1186/s13012-024-01339-x · Cited by 68 · see citing works
Clinical Decision Support: Strategies for Success
Khalifa, M. (2014) · Procedia Computer Science · doi:10.1016/j.procs.2014.08.063 · Cited by 61 · see citing works
“Many studies recommended outsourcing the clinical knowledge management function to third party commercial vendors or other organizations [40, 41].”
BackgroundOr, C. K., Dohan, M. S., & Tan, J. (2014) · Journal of Medical Systems · doi:10.1007/s10916-014-0099-9 · Cited by 53 · see citing works
“Clinical Content pertains to the data describing clinical aspects of a practice, for the purpose of facilitating clinical knowledge management [4, 23], describing patients, their conditions, specialized vocabularies, and data informing clinical decisions [4].”
BackgroundDesign and evaluation of Ubiquitous Information Systems and use in healthcare
Maaß, W., & Varshney, U. (2012) · Decision Support Systems · doi:10.1016/j.dss.2012.08.007 · Cited by 50 · see citing works
“Several technical HIS have failed to meet needs of medical personnel as those lacked understandings of ‘soft issues’ [102].”
BackgroundSittig, D. F., Wright, A. T., Meltzer, S. N., Simonaitis, L., Evans, R. S., Nichol, W. P., Ash, J. S., & Middleton, B. (2011) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-11-13 · Cited by 48 · see citing works
“0 technologies) that allows customers to collaborate asynchronously on CDS content [23]? • Do you have an Internet accessible repository for clients to share their locally generated clinical decision support content with other clients [24]?”
BackgroundSelf-citationElectre Tri-C, a multiple criteria decision aiding sorting model applied to assisted reproduction
Figueira, J. R., Dias, J. A., Matias, S., Roy, B., Carvalho, M. J., & Plancha, C. E. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2010.12.001 · Cited by 49 · see citing works
Palojoki, S., Pajunen, T., Saranto, K., & Lehtonen, L. (2016) · JMIR Medical Informatics · doi:10.2196/medinform.5238 · Cited by 46 · see citing works
“The sociotechnical approach facilitates understanding of the risks in the development, implementation, and use of EHR and HIT while accounting for complex interactions of technology within the health care system [5-12].”
Knowledge management in health: a systematic literature review
Rocha, E. S. B., Nagliate, P. D. C., Furlan, C. E. F. B., Júnior, K. R., Trevizan, M. A., & Mendes, I. A. C. (2012) · Revista Latino-Americana de Enfermagem · doi:10.1590/s0104-11692012000200024 · Cited by 45 · see citing works
“O uso da GC como recurso para apoio à tomada de decisão em saúde tem sido explorado(17-20), com abordagens ao desenvolvimento de sistemas de apoio à decisão, caracterização do estado atual do uso de ferramentas e técnicas de GC em decisão clínica.”
Clinician collaboration to improve clinical decision support: the Clickbusters initiative
McCoy, A. B., Russo, E. M., Johnson, K. B., Addison, B., Patel, N., Wanderer, J. P., Mize, D. E., Jackson, J. G., Reese, T. J., Littlejohn, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac027 · Cited by 41 · see citing works
Self-citationAli, N. B., Whiddett, D., Tretiakov, A., & Hunter, I. M. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.02.011 · Cited by 40 · see citing works
“[8] D.”
BackgroundASHP Statement on the Pharmacist’s Role in Clinical Informatics
(2016) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp150540 · Cited by 40 · see citing works
“13 Knowledge Asset Management.”
Wright, A. T., Ash, J. S., Erickson, J. L., Wasserman, J. A., Bunce, A., Stanescu, A., Hilaire, D. S., Panzenhagen, M., Gebhardt, E., McMullen, C. K., et al. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001771 · Cited by 39 · see citing works
Self-citationOntologies Applied in Clinical Decision Support System Rules: Systematic Review
Jing, X., Min, H., Gong, Y., Biondich, P., Robinson, D., Law, T., Nøhr, C. G., Faxvaag, A., Rennert, L., Hubig, N., et al. (2022) · JMIR Medical Informatics · doi:10.2196/43053 · Cited by 36 · see citing works
“However, the process of developing, updating, and maintaining CDSS rules is time-consuming and resource intensive [4,43], making it difficult for both large institutions and resource-constrained small-scale practices.”
(2010) · PubMed · Cited by 36 · see citing works
Self-citationCollaboration-based medical knowledge recommendation
Huang, Z., Lü, X., Duan, H., & Zhao, C. (2011) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2011.10.002 · Cited by 34 · see citing works
Wright, A. T., Ash, J. S., Aaron, S., Ai, A., Hickman, T.-T. T., Wiesen, J. F., Galanter, W., McCoy, A. B., Schreiber, R., Longhurst, C., et al. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.08.001 · Cited by 31 · see citing works
“However, it has also been shown that developing and maintaining CDS systems can be costly, time-consuming and complex [11-25].”
Self-citationArchambault, P. M., Bilodeau, A., Gagnon, M., Aubin, K., Lavoie, A., Lapointe, J., Poitras, J., Croteau, S., Pham-Dinh, M., & Légaré, F. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1983 · Cited by 31 · see citing works
“For this reason, numerous authors have suggested exploring collaborative Web-based applications to share, create, and update clinical decision support content [19,36,38-40].”
BackgroundMethodSantra, D., Basu, S. K., Mandal, J. K., & Goswami, S. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.113084 · Cited by 31 · see citing works
“Many knowledge representation techniques (Chakraborty (2008); Karabatak and Ince (2009); Sittig et al. 2010; Tanwar et al. 2010a,b; Arsene et al. 2015) are available in the literature.”
BackgroundClinical reasoning in the context of active decision support during medication prescribing
Horsky, J., Aarts, J., Leonie, V., Seger, D. L., van der Sijs, H., & Bates, D. W. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.09.004 · Cited by 30 · see citing works
Singh, R. K., Mathiyazhagan, K., & Gunasekaran, A. (2024) · Journal of Knowledge Management · doi:10.1108/jkm-11-2023-1100 · Cited by 25 · see citing works
McCoy, A. B., Wright, A. T., & Sittig, D. F. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv073 · Cited by 23 · see citing works
Self-citationDixon, B. E., Gamache, R. E., & Grannis, S. J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001514 · Cited by 22 · see citing works
“44 45”
Beyond the threshold: real-time use of evidence in practice
Jones, J. B., Stewart, W. F., Darer, J. D., & Sittig, D. F. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-47 · Cited by 22 · see citing works
“An optimal CDS model should allow knowledge engineers to specify the criteria that govern which interventions are available and the rules that govern the interplay among a trigger, input data, and the intervention [28].”
BackgroundSelf-citationClinical decision support in small community practice settings: a case study
Ash, J. S., Sittig, D. F., Wright, A. T., McMullen, C. K., Shapiro, M. D., Bunce, A., & Blackford Middleton, A. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2010-000013 · Cited by 21 · see citing works
“’ This is a checklist of types of CDS and a list of questions about CDS management.(18) We also received a demonstration of the system via webex.”
Self-citationChang, S., Hsieh, P., & Chen, H.-F. (2015) · Technology and Health Care · doi:10.3233/thc-151087 · Cited by 19 · see citing works
“Some studies claimed that knowledge management in health care is still in its infancy [13] and has yet to be successfully implemented in the health care industry.”
Khajouei, H., & Khajouei, R. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.10.012 · Cited by 19 · see citing works
Monsen, K. A., Neely, C., Oftedahl, G., Kerr, M. J., Pietruszewski, P., & Farri, O. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.06.004 · Cited by 18 · see citing works
Determinants of knowledge management system adoption in health care
Tsai, J. C.-A., & Hung, S. (2016) · Journal of Organizational Computing and Electronic Commerce · doi:10.1080/10919392.2016.1194062 · Cited by 19 · see citing works
“For example, a hospital widely uses web-based viewers, internet-based tools, and enterprise-wide tools to support CDSS functionality (Sittig et al. 2010).”
BackgroundSittig, D. F., & Ash, J. S. (2011) · The Annals of Family Medicine · doi:10.1370/afm.1291 · Cited by 17 · see citing works
Self-citationIntegration of the Enterprise Electronic Health Record and Anesthesia Information Management Systems
Springman, S. R. (2011) · Anesthesiology Clinics · doi:10.1016/j.anclin.2011.05.007 · Cited by 16 · see citing works
A Knowledge Editing Service for Multisource Data Management in Remote Health Monitoring
Colantonio, S., Esposito, M., Martinelli, M., Pietro, G. D., & Salvetti, O. (2012) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2012.2215622 · Cited by 16 · see citing works
“A comprehensive comparison between most of those approaches can be found in [3].”
BackgroundDefinition, Scope, and Challenges
Greenes, R. A. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-398476-0.00001-4 · Cited by 13 · see citing works
Taber, P. A., Staes, C. J., Phengphoo, S., Rocha, E., Lam, A., Fiol, G. D., Maviglia, S. M., & Rocha, R. A. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103852 · Cited by 11 · see citing works
Jabin, M. S. R. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1785086 · Cited by 10 · see citing works
“In the patient safety domain, models such as the sociotechnical framework proposed by Sittig and Singh (2010) and broader systems-based approaches have advanced understanding of how technology-related risks arise from interactions across multiple system dimensions (24).”
Measuring agreement between decision support reminders: the cloud vs. the local expert
Dixon, B. E., Simonaitis, L., Perkins, S. M., Wright, A. T., & Blackford Middleton, A. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-31 · Cited by 9 · see citing works
“KM is challenging as it requires significant investment in human and infrastructure resources to ensure that the knowledge base supporting CDS is accurate and up-to-date [23-25].”
BackgroundSelf-citationJing, X., Min, H., Gong, Y., Sittig, D. F., Biondich, P., Robinson, D., Law, T., Wright, A. T., Nøhr, C. G., Faxvaag, A., et al. (2022) · medRxiv · doi:10.1101/2022.05.11.22274984 · Cited by 9 · see citing works
“process has been recognized by larger institutes [4, 35], which usually are better resourced than small-”
BackgroundSelf-citationYarahuan, J. W., Billett, A. L., & Hron, J. D. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1693123 · Cited by 8 · see citing works
Gestión del conocimiento: experiencias de instituciones académicas y hospitalarias
Torres-Narváez, M.-R., Cruz-Velandia, I., & Jaramillo, J. H. (2014) · Revista Ciencias de la Salud · doi:10.12804/revsalud12.2.2014.03 · Cited by 8 · see citing works
MEDICINE 4.0—Interplay of Intelligent Systems and Medical Experts
Schnurr, H.-P., Aronsky, D., & Wenke, D. (2018) · Progress in IS · doi:10.1007/978-3-319-73546-7_3 · Cited by 8 · see citing works
Carney, T. J., Morgan, G. P., Jones, J., McDaniel, A. M., Weaver, M. T., Weiner, B. J., & Haggstrom, D. A. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.08.005 · Cited by 7 · see citing works
Singh, K., & Wright, A. T. (2015) · Springer eBooks · doi:10.1007/978-3-319-22753-5_6 · Cited by 7 · see citing works
“In a survey of six geographically diverse healthcare systems [ 38 ], all of the organizations surveyed had a dedicated, multidisciplinary team of individuals responsible for creating and maintaining clinical knowledge.”
BackgroundSelf-citationBoosting Quality Registries with Clinical Decision Support Functionality
Wannheden, C., Forsberg, H. H., Eftimovska, E., Westling, K., & Ellenius, J. (2017) · Methods of Information in Medicine · doi:10.3414/me16-02-0030 · Cited by 7 · see citing works
“In such architectures, knowl edge management can be facilitated by web-based tools and repositories that enable collaborative knowledge development, sharing and maintenance [24, 25].”
BackgroundShakibaei, E., Sohal, A. S., Moghadam, V. K., Rajabkhah, K., Prajogo, D. I., Melder, A., Nguyen, Q., Bingham, G., & Tong, E. (2025) · Health Policy and Technology · doi:10.1016/j.hlpt.2025.101014 · Cited by 6 · see citing works
A medical knowledge management system based on expert tagging (MKMST)
Baigorri, A., Villadangos, J., Astráin, J. J., & Córdoba, A. (2013) · WIT transactions on information and communication technologies · doi:10.2495/data130201 · Cited by 6 · see citing works
“[10].”
Kam, H. J., Kim, J. A., Cho, I., Kim, Y. H., & Park, R. W. (2011) · PubMed Central · Cited by 5 · see citing works
Development of a Prospective Web-Based Inventory System for Management of Lab Facilities
Karim, A., Saad, M. F., & Haque, M. S. (2011) · Journal of Emerging Trends in Engineering and Applied Sciences · Cited by 5 · see citing works
From emergence to recoverability: a sociotechnical control trajectory theory of HIT-related risk
Jabin, M. S. R. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1811981 · Cited by 4 · see citing works
“This has implications for procurement, commissioning, and lifecycle management of HIT systems, suggesting that governance arrangements should explicitly address recovery as well as prevention (38, 59, 60).”
Knowledge as a Service at the Point of Care.
Shellum, J. L., Freimuth, R. R., Peters, S. G., Nishimura, R. A., Chaudhry, R., Demuth, S. J., Knopp, A. L., Miksch, T. A., & Milliner, D. S. (2016) · PubMed Central · Cited by 4 · see citing works
Design and Experimental Approach to the Construction of a Human Signal-Molecule-Profiling Database
Zhao, X., & Dong, T. (2013) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph10126887 · Cited by 4 · see citing works
“Many countries plan to invest in health IT research by constructing biobanks and health information databases [1].”
BackgroundKorvesi, V., Chouvarda, I., Mastorakos, G. P., & Goulis, D. G. (2020) · European Journal of Clinical Investigation · doi:10.1111/eci.13291 · Cited by 4 · see citing works
Michel, J. J., Utidjian, L., Karavite, D., Hogan, A. K., Ramos, M., Miller, J. M., Shiffman, R. N., & Grundmeier, R. W. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2016-10-ra-0173 · Cited by 3 · see citing works
O’Sullivan, D. M., Doyle, J., Michalowski, W. J., Szymon Wilk, A., Farion, K. J., & Kuziemsky, C. E. (2011) · PubMed Central · Cited by 3 · see citing works
DesAutels, S. J., Fox, Z. E., Giuse, D. A., Williams, A. M., Kou, Q.-H., Weitkamp, A. O., Patel Neal, R., & Giuse, N. B. (2016) · PubMed Central · Cited by 2 · see citing works
A Content Management System as an Information Management System in Interdisciplinary Research
Rola, P., & Kuchta, D. (2020) · Acta Universitatis Lodziensis Folia oeconomica · doi:10.18778/0208-6018.350.04 · Cited by 3 · see citing works
Clinical Decision Support: It’s More than Just Alerts
Vijayaraghavan, M., Masson, L. M., & Kannry, J. L. (2022) · Springer eBooks · doi:10.1007/978-3-030-93765-2_7 · Cited by 2 · see citing works
Nyatuka, D. R., Macharia, P., & Jabin, M. S. R. (2026) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580261485823 · Cited by 1 · see citing works
“65,77 The results are also consistent with implementation science frameworks such as the NASSS framework, which emphasizes the complexity of adopting and scaling health technologies.”
Liu, W. (2014) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp120351 · Cited by 1 · see citing works
Comparing physician and hospital manager perceptions of medical knowledge management systems success
Chang, S., Hsieh, P., & Chen, H. (2012) · International Conference on Information Technology in Medicine and Education · doi:10.1109/itime.2012.6291401 · Cited by 1 · see citing works
“Some studies have claimed that KM in health care is still in its infancy [9], and is yet to be successfully implemented into the health care industry.”
BackgroundFeasibility study on service-based data acquisition for human signal molecule profiling database
Zhao, X., & Dong, T. (2013) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2013.6701619 · Cited by 1 · see citing works
Improving Stock Management in Healthcares to Increase Efficiency
Çınar, O. (2012) · The journal of academic social science studies/The journal of academic social science · doi:10.9761/jasss_245 · Cited by 1 · see citing works
An Evidence-Based Health Care Knowledge Integration System: Assessment Protocol
Nabelsi, V., & Croteau, S. (2018) · JMIR Research Protocols · doi:10.2196/11754 · Cited by 1 · see citing works
“As such, several authors suggested exploring collaborative Web-based platforms to share, create, and update content of clinical decision-assistance systems [46,50-53].”
BackgroundOr, C. K., Liu, K., & Wang, H. (2018) · Advances in healthcare information systems and administration book series · doi:10.4018/978-1-5225-5460-8.ch018 · Cited by 1 · see citing works
CONCEPTUAL FOUNDATION OF SAFER GUIDES
Sittig, D. F., & Singh, H. (2015) · Apple Academic Press eBooks · doi:10.1201/b18371-5 · Cited by 1 · see citing works
Self-citationInformatics and Decision Support
Dexheimer, J. W., Johnson, L. H., Solti, I., Aronsky, D., & Pestian, J. P. (2012) · Translational bioinformatics · doi:10.1007/978-94-007-5149-1_11 · Cited by 1 · see citing works
Implementation of Laboratory Equipment Loan System in SiDal (BigData Laboratory System)
Asmoro, C. P., Susanti, H., & Maemunah, I. (2024) · EDULAB Majalah Ilmiah Laboratorium Pendidikan · doi:10.14421/edulab.2023.82.07 · Cited by 1 · see citing works
Proctor, S. N., & Desai, B. (2025) · Applied Clinical Informatics · doi:10.1055/a-2620-6221 · Cited by 1 · see citing works
Design Considerations of a Medical Expert System for Differential Diagnosis of Low Back Pain (ESLBP)
Santra, D., Mandal, J. K., Basu, S. K., & Goswami, S. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-2188-1_34 · Cited by 1 · see citing works
Knowledge Integration and Use-Case Analysis for a Customized Drug-Drug Interaction CDS Service
Kam, H. J., Park, M. Y., Kim, W., Yoon, D. Y., Ahn, E. K., & Park, R. W. (2010) · Communications in computer and information science · doi:10.1007/978-3-642-16444-6_7 · Cited by 1 · see citing works
“Clinically inappropriate or excessive numbers of alerts can cause ‘alert fatigue’ for the clinician; this can render clinicians insensitive to alerts, and can degrade their confidence in the alerts, the alerting system and CDS as a whole [ 5 ].”
BackgroundOrlioglu, S., Boobalan, A. S., Abanyie, K., Boyce, R. D., Min, H., Gong, Y., Sittig, D. F., Biondich, P., Wright, A. T., Nøhr, C. G., et al. (2024) · medRxiv · doi:10.1101/2024.09.22.24314152 · Cited by 1 · see citing works
Self-citationKnowledge Delivery and Decision Support for Behavioral Healthcare Professionals
Kennedy, R. S., Weiss, L. G., Dewan, N. A., Luo, J. S., & Lorenzi, N. M. (2010) · Computers in health care · doi:10.1007/978-1-84996-344-2_4
Clinical Commissioning Groups in the UK
Baskaran, V., Johns, S. E., Bali, R. K., Naguib, R. N. G., & Wickramasinghe, N. S. (2011) · International Journal of Healthcare Delivery Reform Initiatives · doi:10.4018/jhdri.2011100104
An Inference Engine for Estimating Outside States of Clinical Test Items
Sakata, M., Yücel, Z., Shinozawa, K., Hagita, N., Imai, M., Furutani, M., & Matsuoka, R. (2013) · ACM Transactions on Management Information Systems · doi:10.1145/2517084
“Test Item Selection In content developments of clinical decision support systems, a large number of health care professionals and information scientists are involved such as pharmacists, clinical analysts, software developers, physicians and administrators [Sittig et al. 2010].”
BackgroundConstruction of a Drug Safety Assurance Information System Based on Clinical Genotyping
Springer, J. A., Iannotti, N. V., Sprague, J. E., & Kane, M. D. (2011) · ISRN Bioinformatics · doi:10.5402/2012/982737
Carney, T. J. (2013) · IUScholarWorks (Indiana University) · doi:10.7912/c2/939
Lin, C. H., Yang, A. W. H., Pittayachawan, S., & Wickramasinghe, N. S. (2016) · Healthcare delivery in the information age · doi:10.1007/978-3-319-25973-4_11
Virtual Communities of Practice in Immersive Virtual Worlds
Majewski, G., & Usoro, A. (2012) · IGI Global eBooks · doi:10.4018/978-1-4666-0312-7.ch019
Kramer, H. S. (2014) · J. Willard Marriott Library · doi:10.26053/0h-wwe2-h7g0
Using Performance Measurement in Healthcare Analytics
Nammour, F. L., Mansour, N., & Danas, K. (2016) · World Congress on Medical Physics and Biomedical Engineering, September 7 - 12, 2009, Munich, Germany · doi:10.1007/978-3-319-32703-7_162
Creating an immunization content database for knowledge management across clinical systems
Wen, H., Mathe, J. L., Weinberg, S. T., Weitkamp, A. O., & Nelson, S. D. (2019) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxz134
Governance for Clinical Decision Support
Wright, A. T., Sittig, D. F., Ash, J. S., Bates, D. W., Feblowitz, J. C., Fraser, G., Maviglia, S. M., McMullen, C. K., Nichol, W. P., Pang, J. E., et al. (2013) · Apple Academic Press eBooks · doi:10.1201/b16306-17
Self-citationA Systematic Review of Ontologies applied in Clinical Decision Support System Rules (Preprint)Preprint
Jing, X., Min, H., Gong, Y., Biondich, P., Robinson, D., Law, T., Nøhr, C. G., Faxvaag, A., Rennert, L., Hubig, N., et al. (2022) · doi:10.2196/preprints.43053
Sagala, S., Prilandita, N., Pranindita, N., Gracia, E., Ikhsanul, M. N., Seba, R. R., Zahra, T. A., & Sitanggang, C. M. (2024) · AIP conference proceedings · doi:10.1063/5.0238339
van Brummelen, N., Leopold, J. H., & Medlock, S. K. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02293-x
“Sittig et al., 2010 surveyed six sites in the United States of America (USA), which used five different tools to facilitate con - tent development.”
Campion, T. R., Tran, T. V., Cheriff, A. D., Gossey, J. T., & Cole, C. L. (2026) · Applied Clinical Informatics · doi:10.1055/a-2880-6252
Historia clínica electrónica de atención primaria de salud: Valoración de las enfermeras
Masclans, J. G. (2014) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
- Authors
- Sittig, D. F., Wright, A., Simonaitis, L., Carpenter, J. D., Allen, G. O., Doebbeling, B. N., Sirajuddin, A. M., Ash, J. S., & Middleton, B.
- Affiliations
- Memorial Hermann (US)
- Regenstrief Institute (US)
- Health Services Research & Development (US)
- Oregon Health & Science University (US)
- Corresponding author
- Sittig, Dean F.
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2010-01
- Volume
- 79
- Issue
- 1
- Pages
- 44-57
- ISSN
- 1386-5056 (ISSN-L)
- PMC
- PMC2895508
- OpenAlex
- W1982907589
- Cited by
- 101
- References
- 39
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 5 · 2022: 5 · 2021: 1 · 2020: 3 · 2019: 3 · 2018: 4 · 2017: 6 · 2016: 10 · 2015: 5 · 2014: 9 · 2013: 13 · 2012: 14
- FWCI
- 21.16
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- International Journal of Medical Informatics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- IUScholarWorks (Indiana University) (submittedVersion)
- Purdue University Indianapolis (Indiana University) (submittedVersion)
- Funders
- Agency for Healthcare Research and Quality
- U.S. National Library of Medicine (R01LM006942, R56LM006942)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Decision Sciences › Management Science and Operations Research › Knowledge Management and Technology
- Keywords
- clinical knowledge management, clinical decision support, collaborative content development
- MeSH terms
- Hospital Information Systems, Humans, Terminology as Topic, Patient Care Team, Software, United States, User-Computer Interface, Information Storage and Retrieval, Medical Records Systems, Computerized, Vocabulary, Controlled, Health Care Surveys, Decision Support Systems, Clinical, Internet, Interdisciplinary Communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
