CoChat
Sign Up
Cited by 825Electronic Health Records SystemsInformation Systems Theories and ImplementationTechnology and Human Factors in Education and Health

Types of Unintended Consequences Related to Computerized Provider Order Entry

Open access
Campbell, E. M., Sittig, D. F., Ash, J. S., Guappone, K. P., Dykstra, R. H.·2006·Journal of the American Medical Informatics Association
DOI 10.1197/jamia.m2042PMC PMC1561794
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: To identify types of clinical unintended adverse consequences resulting from computerized provider order entry (CPOE) implementation. DESIGN: An expert panel provided initial examples of adverse unintended consequences of CPOE. The authors, using qualitative methods, gathered and analyzed additional examples from five successful CPOE sites. METHODS: Using a card sort method, the authors developed a categorization scheme for the 79 unintended consequences initially identified and then iteratively modified the scheme to categorize 245 additional adverse consequences resulting from fieldwork. Because the focus centered on consequences requiring prevention or remedial action, the authors did not further analyze reported unintended beneficial (positive) consequences. RESULTS: Unintended adverse consequences (UACs) fell into nine major categories (in order of decreasing frequency): 1) more/new work for clinicians; 2) unfavorable workflow issues; 3) never ending system demands; 4) problems related to paper persistence; 5) untoward changes in communication patterns and practices; 6) negative emotions; 7) generation of new kinds of errors; 8) unexpected changes in the power structure; and 9) overdependence on the technology. Clinical decision support features introduced many of these unintended consequences. CONCLUSION: Identifying and understanding the types and in some instances the causes of unintended adverse consequences associated with CPOE will enable system developers and implementers to better manage implementation and maintenance of future CPOE projects.

Integrity signals

Registry

Crossref · 10.1197/jamia.m2042

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

7 of 26 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Interpretationcitation

    If patient loads are not decreased temporarily during training periods, clinicians work longer hours to complete their combined electronic and clinical work.

    “If their patient loads are not decreased temporarily during training periods, clinicians work longer hours to complete their combined electronic and clinical work.13”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [13] 13. Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study. Br Med J 2005;331(7528):1313-6. source

      In “Adaptation of the organisation to the new electronic medical record environment” (p. 1314), clinicians report an additional 30–75 minutes of electronic-record work per day, and the implementation provided extra clinicians to help with the initial workload. The paper does not establish the claimed condition that patient loads were not temporarily reduced during training, or that this caused clinicians to work longer hours. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingcitation

    Loss of efficiency from CPOE often recovers over time.

    “This loss of efficiency often recovers over time.12”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [12] 12. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians' time utilization in ambulatory primary care internal medicine prac- tices. J Am Med Inform Assoc 2001;8(4):361-71. source
  • Premisecitation

    CPOE adoption can generate new kinds of health care-practice related errors.

    “Studies have indicated that CPOE adoption can generate new kinds of health care-practice related errors, while others have described roles for CPOE in both preventing and causing medication errors.(4,20 -25 )”
    Quote in the paperSubstantial claim1 of 7 sources checked
    • cites [4] 4. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. J Am Med Inform Assoc 2005;293(10):1197-203. source
    • cites [20] 20. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redun- dant laboratory tests. Am J Med 1999;106(2):144 -50. source
    • cites [21] 21. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6(4):313-21. source
    • cites [22] 22. Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin GA. The epidemiology of prescribing errors: the potential impact of computerized prescriber order entry. Arch Intern Med 2004;164(7):785-92. source
    • cites [23] 23. Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a medication dosing error related to CPOE. J Am Med Inform Assoc 2005;12(4):377-82. source
    • cites [24] 24. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. [see comment]. Arch Intern Med 2003;163(12):1409 -16. source
    • cites [25] 25. Oren E, Shaffer ER, Guglielmo BJ. Impact of emerging technol- ogies on medication errors and adverse drug events. Am J Health-System Pharm 2003;60(14):1447-58. source
  • Findingreasoning

    CPOE systems can add to ineffective or dysfunctional workflows when CPOE developers' computational models do not reflect actual clinical practices.

    “These systems can add to ineffective or dysfunctional workflows when CPOE developers' computational models do not reflect actual clinical practices.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    It is difficult to reconcile old order sets with new institutional initiatives to streamline care processes and follow recent evidence-based clinical practice guidelines.

    “It is also difficult to reconcile old order sets with new institutional initiatives to streamline care processes and to follow the most recent evidence-based clinical practice guidelines.”
    Quote in the paperSubstantial claim
  • May combine two claims. 26
  • Classified as probably not a substantial claim. 12
  • The quote only partly states this claim. 1

  • May combine two claims.

    Interpretationcitation

    Embedding clinical decision support within CPOE systems may increase clinician-users' access to educational material and may affect learning and retention.

    “Although limited experience exists, embedding clinical decision support (CDS) within CPOE systems may increase clinician-users' access to educational material 30,31 and may affect learning and retention.32”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [30] 30. Rosenbloom S, Geissbuhler A, Dupont W, et al. Effect of CPOE user interface design on user-initiated access to educational and patient information during clinical care. J Am Med Inform Assoc 2005;12(4):458 -73. source
    • cites [31] 31. Bochicchio GV, Smit PA, Moore R, et al. Pilot Study of a Web-Based Antibiotic Decision Management Guide. J Am Coll Surg 2006;202(3):459 -67. source
    • cites [32] 32. Knight A, Kravet S, Harper G, Leff B. The effect of computerized provider order entry on medical student clerkship experiences. J Am Med Inform Assoc 2005;12(5):554 -60. source

      Citation 32 is attached to the claim that CPOE may affect learning and retention, not to the preceding claim about access to educational material. The Discussion reports that CPOE may affect medical students’ learning experiences, but the paper does not address retention or establish the effect for clinician-users generally. No page number is available in our copy. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingcitation

    CPOE systems prevent some types of errors while creating or propagating new ones.

    “CPOE systems prevent some types of errors while creating or propagating new ones.4,5,21,22,24,25,33,54 -61”
    Quote in the paperSubstantial claim2 of 10 sources checked
    • cites [4] 4. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. J Am Med Inform Assoc 2005;293(10):1197-203. source
    • cites [5] 5. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11(2):104 -12. source
    • cites [21] 21. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6(4):313-21. source
    • cites [22] 22. Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin GA. The epidemiology of prescribing errors: the potential impact of computerized prescriber order entry. Arch Intern Med 2004;164(7):785-92. source
    • cites [24] 24. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. [see comment]. Arch Intern Med 2003;163(12):1409 -16. source
    • cites [25] 25. Oren E, Shaffer ER, Guglielmo BJ. Impact of emerging technol- ogies on medication errors and adverse drug events. Am J Health-System Pharm 2003;60(14):1447-58. source
    • cites [33] 33. Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized phy- sician order entry. J Am Med Inform Assoc 2004;11(4):270 -7. source

      The Abstract (page unavailable) says CPOE combined with basic clinical decision support decreases the likelihood of serious medication errors. Neither that passage nor the Discussion (page unavailable) says CPOE creates or propagates new errors; the Discussion addresses alerts that clinicians may ignore, not errors caused by CPOE. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [54] 54. Berger RG, Kichak JP. Computerized physician order entry: helpful or harmful? J Am Med Inform Assoc 2004;11(2):100 -3. source
    • cites [55] 55. Hagland M. Safe ways. Hospitals looking to improve patient safety are turning to CPOE, bar coding and e-prescribing. Healthcare Inform 2004;21(8):20 -5.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [56] 56. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients. Pediatrics 2003; 112(3 Pt 1):506 -9. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    Implementation of CPOE requires advanced hardware platforms that can support clinical software, and purchasing or upgrading hardware is an ongoing need.

    “Implementation of CPOE requires advanced hardware platforms that can support clinical software.Purchasing or upgrading hardware is not a one-time event, as future technology advances make this an ongoing need.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Findingreasoning

    All employees must be trained in system use and retrained after substantive system changes.

    “All employees must be trained in system use, and retrained after substantive system changes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    As clinical software systems increase in scope and capabilities, more users require more computer access time via more computers.

    “As clinical software systems increase in scope and capabilities, more users require more computer access time, via more computers.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    When the system fails to support all the individual role-players who must interact with it, work-shifting to others may occur, leading to resentment and ineffective work activity synchronization.

    “When the system fails to support all the individual role-players who must interact with it, work-shifting to others may occur, leading to resentment and ineffective work activity synchronization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    After go-live, ongoing work begins: the system must be tuned, upgraded, tested, interfaced, and backed up regularly.

    “But after go-live, ongoing work begins.The system must be tuned, upgraded, tested, interfaced with other systems, and backed up regularly.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Never-ending system demands arise regarding hardware and software purchases, implementation tasks, and maintenance issues, and represent UACs from the perspective of both administration and IT staff.

    “Never-ending system demands arise regarding hardware and software purchases, implementation tasks, and maintenance issues.They represent UACs from the perspective of both administration and IT staff.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Findingreasoning

    Software application demands are never-ending.

    “Software application demands are also never-ending.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    Unintended adverse consequences (UACs) of CPOE fall into nine major categories, in order of decreasing frequency: more/new work for clinicians; unfavorable workflow issues; never ending system demands; problems related to paper persistence; untoward changes in communication patterns and practices; negative emotions; generation of new kinds of errors; unexpected changes in the power structure; and,

    “Unintended adverse consequences (UACs) fell into nine major categories (in order of decreasing frequency): 1) more/new work for clinicians; 2) unfavorable workflow issues; 3) never ending system demands; 4) problems related to paper persistence; 5) untoward changes in communication patterns and practices; 6) negative emotions; 7) generation of new kinds of errors; 8) unexpected changes in the”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The five selected hospitals are representative of successful CPOE implementations, so the UACs observed there generalize to other successful CPOE sites.
    • Rests on an assumption: The frequency of occurrence of UAC types in the data (Table 2) accurately reflects their relative frequency in CPOE implementations.
    • Rests on an assumption: The UACs observed are caused by CPOE rather than other organizational changes or site-specific factors.
    • Rests on an assumption: The perspectives of clinical end-users, IT staff, and administrators are sufficient to identify the major types of UACs.

  • May combine two claims.

    Findingnone given

    Administrators and researchers commonly leverage CPOE to collect information not directly related to patient care, and the time burden for doing so usually falls on clinicians.

    “Administrators and researchers commonly leverage CPOE to collect information not directly related to patient care.The time burden for doing so usually falls on clinicians.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    As clinical staff increasingly depends on the technology, pressure to keep the system operational 24x7x365 increases, and round-the-clock help-desk support becomes necessary.

    “As the clinical staff increasingly depends on the technology for their daily work, pressure to keep the system operational 24ϫ7ϫ365 increases.Roundthe-clock help-desk support becomes necessary.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Providers suggest that reduction in communication increases the likelihood of errors due to miscommunication, delayed initiation and execution of orders, and fewer team-wide discussions regarding planning and coordination of care.

    “The providers further suggest that reduction in communication increases the likelihood of errors due to miscommunication, delayed initiation and execution of orders, and fewer team-wide discussions regarding planning and coordination of care.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Physicians report loss of professional autonomy when CPOE systems prevent them from ordering preferred tests or medications, force compliance with guidelines, or limit narrative flexibility.

    “Physicians report loss of professional autonomy when CPOE systems prevent them from ordering the types of tests or medications they prefer, or force them to comply with clinical guidelines they may not embrace, or limit their narrative flexibility through structured rather than free-text clinical documentation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Clinical decision support generates a disproportionately large number, over 25%, of UACs, spanning all 9 types.

    “During analysis of unintended consequences, the researchers learned that clinical decision support generates a disproportionately large number (over 25%) of UACs, spanning all 9 types above.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    CPOE-based CDS is not consistently useful for clinicians, is impractical to maintain, and is less than fully reliable in complex situations.

    “Accordingly, CPOE-based CDS is not consistently useful for clinicians, is impractical to maintain, and is less than fully reliable in complex situations.19,24,[33][34][35][36][37][38]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites 19,24,[33][34][35][36][37][38]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingreasoning

    System designs (including poor data organization, data omissions, etc.) and end-user confusion about system functionality contribute to new forms of errors.

    “System designs (including poor data organization, data omissions, etc.), and end-user confusion about system functionality contribute to new forms of errors.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    UACs occur during all CPOE implementations, though not all institutions experience all of the types of UACs.

    “First, UACs occur during all CPOE implementations, though not all institutions experience all of the types of UACs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The project identified hundreds of UACs and grouped them into nine categories.

    “The project identified hundreds of UACs and grouped them into nine categories.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Although consequences of CPOE can be anticipated, their extent is typically underestimated.

    “Although these consequences can be anticipated, their extent is typically underestimated.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Prolonged system failures lasting hours can halt the flow of clinical information so that outpatient activities may be curtailed or canceled and emergency rooms at trauma centers may divert admissions until vital systems are restored.

    “Prolonged system failures (lasting hours) can so dramatically halt the flow of clinical information that outpatient activities may be curtailed or canceled and emergency rooms at trauma centers may divert admissions until vital systems are restored.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Backup systems must operate if the primary system fails.

    “Backup systems must operate if the primary system fails.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    New CPOE-related errors result from problematic electronic data presentations, confusing order option presentations and selection methods, inappropriate text entries, misunderstandings related to test, training, and production versions of the system, and workflow process mismatches.

    “New CPOE-related errors result from: problematic electronic data presentations; confusing order option presentations and selection methods; inappropriate text entries; misunderstandings related to test, training, and production versions of the system; and workflow process mismatches.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Providers used paper for temporary, handwritten data storage for later entry into the computer and as a portable, disposable, computer output display medium for quick reference.

    “We observed providers using paper for temporary, handwritten data storage for later entry into the computer and conversely as a portable, disposable, computer output display medium for quick reference use during their workdays.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Findingnone given

    As a CPOE system evolves, users rely more completely on the software and demand ever more sophisticated functionality for clinical support.

    “As a CPOE system evolves, users rely more completely on the software and demand ever more sophisticated functionality for clinical support.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A wide variety of emotional responses to CPOE were noted, including both strongly negative and highly positive emotions, with negative comments predominating.

    “We noted a wide variety of emotional responses to CPOE, including both strongly negative and highly positive emotions.Negative comments predominated.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Alerts present a major workflow process issue, adding to the steps required to enter an order.

    “Alerts present a major workflow process issue, adding to the steps required to enter an order.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Not all types of UACs may occur with every CPOE implementation.

    “Not all types of UACs may occur with every CPOE implementation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Most often it is the physician who loses power in CPOE-related power changes.

    “Most often it is the physician who loses power: this must be recognized and dealt with explicitly during the CPOE planning process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Clinicians report inappropriate alerts as highly frustrating nuisances.

    “Clinicians report inappropriate alerts as highly frustrating nuisances.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Whenever there are adjustments, there will be unintended consequences.

    “Whenever there are adjustments, there will be unintended consequences.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Paper-based clinical record storage will become obsolete, but use of paper in the clinical setting will not.

    “Paper-based clinical record storage will become obsolete, but use of paper in the clinical setting will not.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Paper remains the most malleable, flexible, and easily transportable data medium available.

  • May combine two claims.

    Interpretationreasoning

    Order standardization may confuse clinicians and forced awkward terminology may compromise patient care.

    “While order standardization may benefit the organization, it may confuse clinicians.”
    • Also stated in: § CPOE system configurations control who may do what (and when) through the use of clinical, role-based authorizations. · hedged
      “Such terminology not only disrupts clinicians' workflows, but might also compromise patient care.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    UACs are the result of a constellation of factors.

    “It is important to view UACs as the result of a constellation of factors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Building and maintaining an appropriate set of CPOE alerts based on an up-to-date evidence base is an onerous, never-ending task.

    “Building and maintaining an appropriate set of CPOE alerts based on an up-to-date evidence base is an onerous, never-ending task.”
    Quote in the paperNot a substantial claim
35 of 35

  • The paper relies on this without support.

    The five selected hospitals are representative of successful CPOE implementations, so the UACs observed there generalize to other successful CPOE sites.

    “The initial expert panel helped the authors to identify five hospitals in three different organizations where successful CPOE implementation occurred (See Table 1).To study a range of successful CPOE implementations, the project selected both academic and non-academic hospitals of differing sizes and geographic locations.”

  • The paper relies on this without support.

    The frequency of occurrence of UAC types in the data (Table 2) accurately reflects their relative frequency in CPOE implementations.

    “Table 2 lists the UACs and their frequencies of occurrence.”

  • The paper relies on this without support.

    The UACs observed are caused by CPOE rather than other organizational changes or site-specific factors.

    “To identify types of clinical unintended adverse consequences resulting from computerized provider order entry (CPOE) implementation.”

  • The paper relies on this without support.

    The perspectives of clinical end-users, IT staff, and administrators are sufficient to identify the major types of UACs.

    “Specifically, we gathered perspectives regarding CPOE from three groups: clinical end-users, IT staff, and administrators.”

  • The paper relies on this without support.

    Paper remains the most malleable, flexible, and easily transportable data medium available.

    “Furthermore, paper remains the most malleable, flexible, and easily transportable data medium available.”

  • The paper relies on this without support.

    CPOE workstation screens cannot display large amounts of clinical data simultaneously.

    “However, CPOE workstation screens cannot display large amounts of data simultaneously.”

  • The paper relies on this without support.

    Health care is increasingly dependent on technology and this is unlikely to change.

    “Health care is increasingly dependent on technology and this is unlikely to change.”

  • The paper relies on this without support.

    Emotional responses to change are inevitable.

    “Emotional responses to change are inevitable.”

  • The paper relies on this without support.

    Currently, CPOE technology is immature and rapidly evolving.

    “Currently, CPOE technology is immature and rapidly evolving.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Research works printed but not linked: 9

  1. 2004, J HEALTHCARE INFORM MANAG, 18, 36
  2. 2004, Medinfo, 11, 1018
  3. 2002, J HEALTHCARE INFORM MANAG, 16, 16
  4. 2004, PHYSIC EXEC, 30, 28
  5. 2005, J HEALTHCARE INFORM MANAG, 19, 70
  6. 2001, Medinfo, 10, 1207
  7. Addison, 2005, Journal of AHIMA / American Health Information Management Association, 76, 64A
  8. 2004, HEALTHCARE INFORM, 21, 20
  9. Kremsdorf, 2005, Health management technology, 26, 66

Cited by

Sorted by citations

Related

Authors
Campbell, E. M., Sittig, D. F., Ash, J. S., Guappone, K. P., & Dykstra, R. H.
Affiliations
Oregon Health & Science University (US)
Corresponding author
Campbell, E. M.
ORCID
Venue
Journal of the American Medical Informatics Association
Publisher
Oxford University Press (OUP)
Published
2006-09-01
Volume
13
Issue
5
Pages
547-556
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
DOI
10.1197/jamia.m2042
PMC
PMC1561794
OpenAlex
W2160382101
Semantic Scholar
90391845572d1d2d9424cddcbdfdb4867be5e69c
Cited by
823
References
61
Citations by year
2026: 9 · 2025: 18 · 2024: 24 · 2023: 29 · 2022: 29 · 2021: 27 · 2020: 27 · 2019: 59 · 2018: 32 · 2017: 44 · 2016: 48 · 2015: 48 · 2014: 52 · 2013: 80 · 2012: 46
FWCI
86.07
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • National Institutes of Health
  • U.S. Public Health Service (ASMM10031)
  • U.S. National Library of Medicine (R01LM006942, LM06942, R56LM006942)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
Keywords
computerized physician order entry, clinical workflow, clinical decision support, clinician workload, communication patterns, medical errors, health information technology implementation
MeSH terms
Communication, Humans, Organizational Innovation, Organizational Culture, Workload, Medical Errors, Medical Order Entry Systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review