The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; none of the 50 sources its claims cite is checked for support yet.
An analysis of electronic health record-related patient safety concerns
Open accessAbstract
OBJECTIVE: A recent Institute of Medicine report called for attention to safety issues related to electronic health records (EHRs). We analyzed EHR-related safety concerns reported within a large, integrated healthcare system. METHODS: The Informatics Patient Safety Office of the Veterans Health Administration (VA) maintains a non-punitive, voluntary reporting system to collect and investigate EHR-related safety concerns (ie, adverse events, potential events, and near misses). We analyzed completed investigations using an eight-dimension sociotechnical conceptual model that accounted for both technical and non-technical dimensions of safety. Using the framework analysis approach to qualitative data, we identified emergent and recurring safety concerns common to multiple reports. RESULTS: We extracted 100 consecutive, unique, closed investigations between August 2009 and May 2013 from 344 reported incidents. Seventy-four involved unsafe technology and 25 involved unsafe use of technology. A majority (70%) involved two or more model dimensions. Most often, non-technical dimensions such as workflow, policies, and personnel interacted in a complex fashion with technical dimensions such as software/hardware, content, and user interface to produce safety concerns. Most (94%) safety concerns related to either unmet data-display needs in the EHR (ie, displayed information available to the end user failed to reduce uncertainty or led to increased potential for patient harm), software upgrades or modifications, data transmission between components of the EHR, or 'hidden dependencies' within the EHR. DISCUSSION: EHR-related safety concerns involving both unsafe technology and unsafe use of technology persist long after 'go-live' and despite the sophisticated EHR infrastructure represented in our data source. Currently, few healthcare institutions have reporting and analysis capabilities similar to the VA. CONCLUSIONS: Because EHR-related safety concerns have complex sociotechnical origins, institutions with long-standing as well as recent EHR implementations should build a robust infrastructure to monitor and learn from them.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Analysis of 100 closed VA investigations of EHR-related safety concerns found that most involved unsafe technology and complex interactions across sociotechnical dimensions. Four recurring concern types accounted for 94% of incidents, showing that EHR safety concerns persist long after go-live even in a mature EHR system.
Key findings
- Of 100 consecutive, unique, closed investigations from August 2009 to May 2013, 74 involved unsafe technology (phase 1), 25 involved unsafe use of technology (phase 2), and 1 involved lack of monitoring (phase 3); 70% involved two or more sociotechnical model dimensions.
- Four concern types accounted for 94% of incidents: unmet display needs (n=36), software modifications/upgrades (n=24), system–system interface failures (n=17), and hidden dependencies in distributed systems (n=17).
- The most commonly involved sociotechnical dimensions were hardware/software (76), clinical content (38), workflow/communication (35), human–computer interface (29), and people (20).
- Incidents came from 55 unique VA facilities; IPS safety scores were low for 48 investigations, intermediate for 38, and high for 14.
Methods
- Retrospective qualitative analysis of completed EHR-related safety investigations from the Veterans Health Administration Informatics Patient Safety Office, using 100 consecutive unique closed investigations extracted from 344 reported incidents.
- Used framework analysis guided by Sittig and Singh’s eight-dimension sociotechnical model and three EHR safety phases; two authors independently indexed concerns, resolving discrepancies by consensus.
- Coded narrative investigation reports in Atlas.ti V.6.2 and arranged data in Microsoft Excel; no inter-rater reliability was reported because the aim was descriptive.
Limitations
- The paper states all incidents involved the same EHR within a single, albeit very large, healthcare system; findings may not generalize to institutions with different organizational characteristics, EHR infrastructure, or reporting mechanisms.
- The paper states the data ranged from unsafe conditions to patient harm, did not analyze additional patient outcome data, and that reporting captures fewer than 10% of medical errors, so prevalence rates cannot be calculated.
- The paper notes self-reported data may be limited by recall bias or reporter knowledge; evident from the text, the retrospective voluntary reporting system and exclusion of duplicate or comingled-record concerns may underrepresent the full universe of EHR-related safety concerns.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
36 Own result · 19 Not checked yet · 15 Own interpretation · 11 Own method · 7 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These incidents also spanned both phase 1 (11 incidents) and phase 2 (6 incidents) of EHR safety.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concern of hidden dependencies in distributed systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across the 36 concerns within this concern type, the contributory dimensions were hardware and software (22 incidents), human–computer interface (22 incidents), workflow and communication (10 incidents), clinical content (9 incidents), people (9 incidents), organizational policies and procedures (2 incidents), and system measurement and monitoring (1 incident).”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One-quarter were classified as phase 2 (ie, unsafe EHR use) and most commonly involved the dimensions of people, clinical content, workflow and communication, and human–computer interface.”
§ An analysis of electronic health record-related patient safety concerns > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, a pharmacist made a data entry error while approving the order for a larger-than-usual amount of diuretic.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All four types of safety concerns affected, or had the potential to affect, multiple patients, although we did not further analyze outcome data except as noted below.”
§ An analysis of electronic health record-related patient safety concerns > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sociotechnical dimensions of phase 1 concerns most commonly involved hardware and software, workflow and communication, and clinical content.”
§ An analysis of electronic health record-related patient safety concerns > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The over-ride released the incorrect dose for administration by nursing staff.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Owing to diminished user confidence in the warning's reliability, the warning was over-ridden.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most (22 of 36) of these concerns were classified as phase 1 issues, followed by 13 phase 2, and one phase 3.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Safety concerns involving hidden dependencies and system–system interfaces are not mutually exclusive.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concern of hidden dependencies in distributed systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 26
- Probably not a substantial claim. 20
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the reports documented a variety of EHR-related safety incidents, four broad types of safety concerns were prominent: unmet data display needs within the EHR; problems with software modifications or upgrades; concerns related to system–system interfaces; and hidden dependencies within the EHR.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Since these printers were installed and configured before recruitment of the current staff, the configuration error was not immediately recognized.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to both intended and unintended software modifications
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Another concern involved ‘legacy’ software (ie, an older system that has not evolved despite newer technologies50) that needed an upgrade or maintenance, but support staff did not have sufficient knowledge of these systems.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to both intended and unintended software modifications
What the sources say
- Not checked yetcites [50] Brodie ML Stonebraker M . Migrating legacy systems: gateways, interfaces & the incremental approach. Morgan Kaufmann Publishers Inc., 1995.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The selected incidents were reported from 55 unique VA facilities.”
§ An analysis of electronic health record-related patient safety concerns > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We classified these concerns into four types: unmet display needs in the EHR, safety concerns with software modifications or upgrades, concerns related to data transmission at system–system interfaces, and concerns of ‘hidden dependencies’ in distributed systems (ie, when one EHR component unexpectedly or unknowingly is affected by the state or condition of another).”
§ An analysis of electronic health record-related patient safety concerns > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A majority (70%) involved two or more model dimensions.”
- Also stated in: § Results · plain
“Incidents frequently reflected occurrence of more than one sociotechnical dimension: 40 incidents were classified with two sociotechnical dimensions, 23 incidents had three, and seven involved four dimensions.”
§ An analysis of electronic health record-related patient safety concerns > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Incidents in this category were also noted to be largely dependent on multiple interactions between these dimensions, and only one incident was coded with a single dimension.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concern of hidden dependencies in distributed systems
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Patient context-related concerns were caused by network failures, conflicts created by non-EHR software, and EHR upgrades that were not compliant with context protocols.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to system–system interfaces
plaindirectn = 17- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This concern type was most often associated with phase 1 EHR safety (21 incidents). Three concerns were classified as phase 2, and none were phase 3.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to both intended and unintended software modifications
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As highlighted in this example, the system–system interface concern involved interactions from multiple sociotechnical dimensions: hardware and software (17 incidents), workflow (6 incidents), and content (5 incidents).”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to system–system interfaces
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The concerns of hidden dependencies primarily involved the dimensions of hardware and software (14 of 17 incidents), workflow (14 incidents), clinical content (9 incidents), and people (5 incidents).”
§ An analysis of electronic health record-related patient safety concerns > Results > Concern of hidden dependencies in distributed systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although a dose error warning appeared on order entry, this particular warning was known to have a high false positive rate.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the IPS-assessed safety scores, 48 investigations were scored low, 38 intermediate, and 14 were scored high.”
§ An analysis of electronic health record-related patient safety concerns > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The nurse, unaware of the discrepancy between the prescribed amount and the amount approved by the pharmacist, administered the larger dose.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“For instance, we previously developed a sociotechnical model that proposes eight interdependent dimensions that are essential to understand EHR-related safety (table 1; Sittig and Singh model).24 34”
§ An analysis of electronic health record-related patient safety concerns > Background and significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The model accounts for the complexities of technology, its users, the involved workflow, and the larger external or organizational policies and context in assessment of EHR-related safety concerns.35 36”
§ An analysis of electronic health record-related patient safety concerns > Background and significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Investments in health information technology (HIT) can enhance the safety and efficiency of patient care and enable knowledge discovery.1”
§ An analysis of electronic health record-related patient safety concerns > Background and significance
What the sources say
- Not checked yetcites [1] Institute of Medicine (IOM). Crossing the quality chasm a new health system for the 21st century. The National Academy Press, 2001.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Members of our multidisciplinary project team, whose areas of expertise included clinical medicine, patient safety, informatics, human factors, and information technology, provided guidance on the content of the themes.”
§ An analysis of electronic health record-related patient safety concerns > Methods > Data analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As compared with system–system interfaces, hidden dependencies are unknown, and therefore potential points of failure and possible safety concerns may not be readily identified.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concern of hidden dependencies in distributed systems
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This example demonstrates how a complex interaction of a technical dimension with non-technical dimensions can potentially develop into a safety concern.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to both intended and unintended software modifications
boostedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result of this poor fit, the displayed information available to the end user failed to reduce uncertainty or led to increased potential for patient harm.”
§ An analysis of electronic health record-related patient safety concerns > Results > Concerns related to unmet data display needs in the EHR
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Given the rich nature of the qualitative data, we believed this number was both valuable and feasible.”
§ An analysis of electronic health record-related patient safety concerns > Methods > Data collection
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“We do not report inter-rater reliability because the aim of the study was to describe the nature and context of common EHR-related safety concerns, not solely classification according to the sociotechnical models.47 49”
§ An analysis of electronic health record-related patient safety concerns > Methods > Data analysis
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, a completed investigation provides additional contextual details pertinent to the incident under study as compared with traditional event reports.”
§ An analysis of electronic health record-related patient safety concerns > Methods > Design and setting
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The two coding authors (ie, a physician with informatics training and a human factors engineer) independently indexed each safety concern after reviewing the results of the IPS investigation. Discrepancies in coding were then resolved by consensus.”
§ An analysis of electronic health record-related patient safety concerns > Methods > Data analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Although these are known safety concerns,43 they were excluded because they are not routinely analyzed by IPS and are handled primarily by a separate office in the VA.”
§ An analysis of electronic health record-related patient safety concerns > Methods > Data collection
What the sources say
- Not checked yetcites [43] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, two authors (DWM and MWS) independently reviewed and summarized the investigation reports to become familiar with the data, but at this secondary stage of analysis, we made no further effort to replicate the safety concern or investigation, determine additional causes, or offer additional solutions.”
§ An analysis of electronic health record-related patient safety concerns > Methods > Data analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite capturing a low percentage of errors, we were able to gain insight about non-technical aspects of EHR-related safety concerns that may not be routinely considered in technology-focused investigations.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, our findings may not represent all types of EHR-related safety concerns and might not be generalizable to other institutions with different organizational characteristics, EHR infrastructure, or patient safety reporting mechanisms.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our findings underscore the importance of continuing the process of detecting and addressing safety concerns long after EHR implementation and ‘go-live’ has occurred.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the analysis of unsafe conditions or near misses is useful to illustrate concerns in EHR-enabled care, we acknowledge that their circumstances or implications may be different from adverse events that result in patient harm.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In general, fewer than 10% of medical errors are captured through reporting, and such data do not allow us to calculate prevalence rates.56 59 60”
§ An analysis of electronic health record-related patient safety concerns > Discussion
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The data used for our analysis were composed of safety concerns that ranged from unsafe conditions to patient harm.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All four emergent safety concerns affected, or had the potential to affect, multiple patients, but we did not analyze additional data on patient outcomes as a result of these concerns.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Additional strengths of our study include the nationwide distribution of our sample of EHR-related safety incidents and the relatively sophisticated implementation and use of the EHR across the VA healthcare system.39”
§ An analysis of electronic health record-related patient safety concerns > Discussion
What the sources say
- Not checked yetcites [39] Spetz J Burgess JF Phibbs CS . What determines successful implementation of inpatient information technology systems? Am J Manag Care 2012;18:157–62.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As an early adopter of EHRs, the VA has evolved into a ‘learning system’ that dedicates resources to investigating safety concerns and making EHR-related safety improvements decades after first launch.42”
§ An analysis of electronic health record-related patient safety concerns > Discussion
What the sources say
- Not checked yetcites [42] Taylor L Wood SD Chapman R . Analysis and Mitigation of Reported Informatics Patient Safety Adverse Events at the Veterans Health Administration. Human Factors and Ergonomics Society, 2012. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the sample size is smaller than that of some other studies, the case descriptions were rich (ie, 2–4 single-spaced pages), spanned a period of 3 years, and represented a continuum of care from home-based primary care to large, urban medical centers.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our study suggests that technology-based solutions alone will only partially mitigate concerns and that interventions to improve EHR-related safety should encompass the people, organizations, systems, and policies that influence how EHRs are used.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safety concerns we identified had complex sociotechnical origins and would need multifaceted strategies for improvement.”
§ An analysis of electronic health record-related patient safety concerns > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper relies on this without support.
Currently, few healthcare institutions have reporting and analysis capabilities similar to the VA.
“Currently, few healthcare institutions have reporting and analysis capabilities similar to the VA.”
The paper relies on this without support.
Few healthcare systems have robust reporting and analytic infrastructure similar to the VA's Informatics Patient Safety Office.
“However, few healthcare systems have robust reporting and analytic infrastructure similar to the VA's IPS.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This study has also added to an emerging but limited UK evidence base that has reported potential negative impacts of electronic records on the doctor-patient relationship [35] and patient safety in secondary care [36–38].”
From Exploring the implementation of an electronic record into a maternity unit: a qualitative study using Normalisation Process Theory (2017) · cites this paper as [36]
Citations
Cites
Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-117 · Cited by 10,865
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,779
Qualitative research in health care: Analysing qualitative data
Pope, C. (2000) · BMJ · doi:10.1136/bmj.320.7227.114 · Cited by 8,131
The SAGE Encyclopedia of Qualitative Research Methods
Given, L. (2008) · doi:10.4135/9781412963909 · Cited by 6,134
Qualitative methods for health research
Green, J., & Thorogood, N. (2009) · Choice Reviews Online · doi:10.5860/choice.47-0901 · Cited by 4,612
Qualitative Research in Health Care
(2006) · Wiley eBooks · doi:10.1002/9780470750841 · Cited by 2,747
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126
Leavitt, M. (2001) · PubMed · Cited by 1,835
Work system design for patient safety: the SEIPS model
Carayon, P., Schoofs Hundt, A., Karsh, B.-T., Gurses, A. P., Alvarado, C. J., Smith, M., & Flatley Brennan, P. (2006) · Quality in Health Care · doi:10.1136/qshc.2005.015842 · Cited by 1,795
Self-citationAsh, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789
Research works printed but not linked: 1
- Institute of Medicine (IOM). Health IT and Patient Safety: Building Safer Systems for Safer Care. Washington, DC: The National Academies Press, 2012.
Cited by
Sorted by citations
Electronic Health Records: Then, Now, and in the Future
Evans, R. S. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s006 · Cited by 816
“Mounting pressure will require EHR implementations to build a robust infrastructure to monitor and learn from adverse events, errors, and potential adverse events and errors [204].”
Kim, M. O., Coiera, E., & Magrabi, F. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw154 · Cited by 263
“System configuration issues were also linked to software updates, eg, decision support errors following updates to a drug database.(26) Sociotechnical contextual variables that contributed to information errors were identified in 71% of studies (n1⁄424).”
BackgroundReport of the AMIA EHR-2020 Task Force on the status and future direction of EHRs
Payne, T. H., Corley, S., Cullen, T. A., Gandhi, T. K., Harrington, L., Kuperman, G. J., Mattison, J. E., McCallie, D. P., McDonald, C. J., Tang, P. C., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv066 · Cited by 220
Current challenges in health information technology–related patient safety
Sittig, D. F., Wright, A. T., Coiera, E. W., Magrabi, F., Ratwani, R. M., Bates, D. W., & Singh, H. (2018) · Health Informatics Journal · doi:10.1177/1460458218814893 · Cited by 183
“55 This must change if we are to gather enough data to identify common modes of failure and estimate the likelihood of similar future events.”
BackgroundSelf-citationSingh, H., & Sittig, D. F. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004486 · Cited by 167
Self-citationCoiera, E., Kocaballi, B., Halamka, J., & Laranjo, L. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0066-9 · Cited by 132
Cohen, G. R., Friedman, C. P., Ryan, A. M., Richardson, C. R., & Adler-Milstein, J. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05025-3 · Cited by 117
An analysis of electronic health record–related patient safety incidents
Palojoki, S., Mäkelä, M., Lehtonen, L., & Saranto, K. (2016) · Health Informatics Journal · doi:10.1177/1460458216631072 · Cited by 120
Sittig, D. F., Belmont, E., & Singh, H. (2017) · Healthcare · doi:10.1016/j.hjdsi.2017.06.004 · Cited by 103
Self-citationThe Impact of Electronic Health Records on Diagnosis
Graber, M. L., Byrne, C., & Johnston, D. (2017) · Diagnosis · doi:10.1515/dx-2017-0012 · Cited by 108
Electronic Health Record–Related Events in Medical Malpractice Claims
Graber, M. L., Siegal, D., Riah, H., Johnston, D., & Kenyon, K. (2015) · Journal of Patient Safety · doi:10.1097/pts.0000000000000240 · Cited by 106
Singh, H., & Sittig, D. F. (2020) · Annals of Internal Medicine · doi:10.7326/m19-0879 · Cited by 77
“This reporting guideline, which we call the Safety-related EHR Research (SAFER) Reporting Framework, can help guide successful implementation (19–25), has been used to understand the dynamic nature of relationships between health IT (including computerized provider order entry, clinical decision…”
MethodSelf-citationKutney-Lee, A. M., Sloane, D. M., Bowles, K. H., Burns, L. R., & Aiken, L. H. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1678551 · Cited by 77
Information Quality Frameworks for Digital Health Technologies: Systematic Review
Fadahunsi, K. P., O’Connor, S., Akinlua, J. T., Wark, P. A., Gallagher, J., Carroll, C., Car, J., Majeed, A., & O’Donoghue, J. (2021) · Journal of Medical Internet Research · doi:10.2196/23479 · Cited by 86
“However, the clinical use of digital information for diagnostic, therapeutic, and prognostic purposes has multiple patient safety problems, including significant harms and death, some of which result from poor information quality (IQ) [6-9].”
BackgroundRate of electronic health record adoption in South Korea: A nation-wide survey
Kim, Y., Jung, K., Park, Y., Shin, D., Cho, S. Y., Yoon, D., & Park, R. W. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.02.009 · Cited by 63
Ten key considerations for the successful optimization of large-scale health information technology
Cresswell, K. M., Bates, D. W., & Sheikh, A. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw037 · Cited by 70
“In relation to technical aspects, for instance, continued measurement and monitoring of technical functioning, including system reliability, response time, upgrades, and system-induced errors, can help in anticipating and mitigating risks.(47,48) Some useful guidance in this respect can be found in a recent conceptual…”
BackgroundCresswell, K. M., Lee, L., Mozaffar, H., Williams, R., Sheikh, A., & on behalf of the NIHR ePrescribing Programme Team (2017) · Health Services Research · doi:10.1111/1475-6773.12581 · Cited by 64
“health system concluded that “EHR-related safety concerns involving both unsafe technology and unsafe use of technology persist long after ‘go-live’” (Meeks et al. 2014).”
BackgroundRobotics Technology in Mental Health Care
Riek, L. D. (2016) · Artificial Intelligence in Behavioral and Mental Health Care · doi:10.1016/b978-0-12-420248-1.00008-8 · Cited by 71
“This has been a growing problem in treatment and safety across other healthcare disciplines, and there is all the more reason to suspect this problem would be magnified for a profession centered on face-to-face interaction (cf. Levinson, Gorawara-Bhat, & Lamb, 2012; Meeks et al., 2014; Montague & Asan, 2014).”
BackgroundLippke, S., Wienert, J., Keller, F. M., Derksen, C., Welp, A., Kötting, L., Hofreuter-Gätgens, K., Müller, H., Louwen, F., Weigand, M., et al. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4579-y · Cited by 75
Abraham, J., Novak, L. L., Reynolds, T. L., Gettinger, A., & Zheng, K. (2016) · Yearbook of Medical Informatics · doi:10.15265/iy-2016-036 · Cited by 61
“The third study, by Meeks et al. (2015), analyzed incidents tracked by a non-punitive, voluntary reporting system used at the U.S. Veterans Health Administration [122].”
Toward Designing Information Display to Support Critical Care
Dunbar, S., Macpherson, B. C., Moretti, E., Fiol, G. D., Bolte, J., Taekman, J. M., Segall, N., & Wright, M. C. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2016-03-ra-0033 · Cited by 56
“Inefficiencies and the opportunity to miss key information make electronic health care information presentation a crucial patient safety priority [8].”
BackgroundDigital technologies: An exploratory study of their role in the resilience of healthcare services
da Rosa, V. M., Saurin, T. A., Tortorella, G. L., Fogliatto, F. S., Tonetto, L. M., & Samson, D. A. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103517 · Cited by 70
Salahuddin, L. B., & Ismail, Z. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.07.004 · Cited by 66
“nadequate knowledge of the health IT by technical support staff an potentially develop safety risks [89].”
BackgroundInfluentialDunn Lopez, K., Chin, C.-L., Leitão Azevedo, R. F., Kaushik, V., Roy, B., Schuh, W., Banks, K., Sousa, V., & Morrow, D. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103359 · Cited by 62
Electronic Health Records and Improved Patient Care: Opportunities for Applied Psychology
Ratwani, R. M. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721417700691 · Cited by 70
“The poor design of EHRs is largely recognized as a major source of these challenges (Meeks et al., 2014; Zahabi, Kaber, & Swangnetr, 2015).”
BackgroundDalal, A. K., Fuller, T. E., Garabedian, P., Ergai, A., Balint, C., Bates, D. W., & Benneyan, J. C. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz002 · Cited by 58
The biopsychosociotechnical model: a systems-based framework for human-centered health improvement
Card, A. J. (2022) · Health Systems · doi:10.1080/20476965.2022.2029584 · Cited by 79
“…access to doctors’ clinical notes may improve patients’ understanding, adherence, and self-care (Walker et al., 2015) Although they were adopted in part to improve safety, EHRs can also contribute to patient harm (P Carayon et al., 2017; Graber et al., 2019; Meeks et al., 2014; Menon et al., 2014)”
BackgroundSafety huddles to proactively identify and address electronic health record safety
Menon, S., Singh, H., Giardina, T. D., Rayburn, W. L., Davis, B. P., Russo, E. M., & Sittig, D. F. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw153 · Cited by 48
Self-citationWright, M. C., Borbolla, D. A., Waller, R. G., Fiol, G. D., Reese, T., Nesbitt, P., & Segall, N. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.yjbinx.2019.100041 · Cited by 48
“However, the ability of information displays to support these activities has been criticized [3–5], and their use has been linked with errors that compromise patient safety [6,7].”
BackgroundMartin, G., Ghafur, S., Cingolani, I., Symons, J., King, D., Arora, S., & Darzi, A. W. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30057-3 · Cited by 51
Rahman Jabin, M. S., Steen, M., Wepa, D., & Bergman, P. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231174307 · Cited by 44
Scantlebury, A., Sheard, L., Watt, I., Cairns, P., Wright, J., & Adamson, J. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0406-0 · Cited by 41
Evidence-based Health Informatics: How Do We Know What We Know?
Ammenwerth, E. (2015) · Methods of Information in Medicine · doi:10.3414/me14-01-0119 · Cited by 47
“Best practice examples are already available [83].”
Automation to optimise physician treatment of individual patients: examples in psychiatry
Bauer, M., Monteith, S., Geddes, J., Gitlin, M. J., Grof, P., Whybrow, P. C., & Glenn, T. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30041-0 · Cited by 44
Barnett, M. L., Mehrotra, A., & Jena, A. B. (2016) · BMJ · doi:10.1136/bmj.i3835 · Cited by 47
Building a learning health system using clinical registers: a non-technical introduction
Øvretveit, J., Nelson, E. C., & James, B. C. (2016) · Journal of Health Organization and Management · doi:10.1108/jhom-06-2016-0110 · Cited by 42
Rahman Jabin, M. S., & Pan, D. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231203600 · Cited by 41
“Six incidents were identi fi ed under this theme (IR 2,5,6,17,18,22 ).”
BackgroundJabin, M. S. R. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1367431 · Cited by 43
Unintended adverse consequences of a clinical decision support system: two cases
Stone, E. G. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx096 · Cited by 39
“There have been concerns voiced in the past about the unintended adverse consequences (UACs) of aspects of health IT, including computerized physician order entry (CPOE) and electronic health records (EHRs).(30) Studies have also been published specifically addressing UACs in CDSSs.”
BackgroundOncologists’ Views Regarding the Role of Electronic Health Records in Care Coordination
Asan, O., Nattinger, A. B., Gürses, A. P., Tyszka, J. T., & Yen, T. W. F. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.17.00118 · Cited by 38
Nurse workarounds in the electronic health record: An integrative review
Fraczkowski, D., Matson, J., & Lopez, K. D. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa050 · Cited by 44
“Usability, defined by Nielsen to include the 5 attributes of (1) easy to learn and (2) remember, (3) efficient to use, (4) has few errors, and (5) is subjectively pleasing,(73) has emerged as a critical problem in health care that increases chance of errors across an array of health information technologies.(18,74,75)…”
BackgroundUnintended Consequences of EHR Systems: A Narrative Review
Vanderhook, S., & Abraham, J. (2017) · Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care · doi:10.1177/2327857917061048 · Cited by 37
“Meeks et al. (2014) noted when a sociotechnical dimension was involved in a safety incident, there was an interaction by another dimension.”
BackgroundInfluentialScott, I. A., Sullivan, C. M., & Staib, A. N. (2018) · Australian Health Review · doi:10.1071/ah17153 · Cited by 47
Barrett, A. K., & Stephens, K. K. (2016) · Health Communication · doi:10.1080/10410236.2016.1196422 · Cited by 35
“These front-line implementers of EHR systems are being studied in medical, nursing, and medical information technology journals (e.g., Ford, Menachemi, Peterson, & Huerta, 2009; Goh, Gao, & Agarwal, 2011; Meeks et al., 2014; Menachemi & Collum, 2011; Noyes, Lewis, Bennett, Widdas, & Brombley, 2013; Stutzer & Rushton,…”
BackgroundInfluentialRahman Jabin, M. S., & Hammar, T. (2022) · DIGITAL HEALTH · doi:10.1177/20552076221131139 · Cited by 32
“46 If these challenges are left undetected even for a shorter period, it can cause severe consequences for multiple patients and widespread impact at a system level.”
BackgroundWright, A. T., Ash, J. S., Aaron, S., Ai, A., Hickman, T.-T. T., Wiesen, J. F., Galanter, W., McCoy, A. B., Schreiber, R., Longhurst, C., et al. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.08.001 · Cited by 31
“It has also been effective in informatics research [47, 48].”
Self-citationPowell, L., Sittig, D. F., Chrouser, K. L., & Singh, H. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.6752 · Cited by 34
Self-citationTurner, P. A., Kushniruk, A. W., & Nøhr, C. (2017) · Yearbook of Medical Informatics · doi:10.15265/iy-2017-014 · Cited by 36
“…implementation, and evaluation of these systems, we have long understood that most challenges result from a failure to design for the complexity of socio-technical systems in healthcare and/or a failure to understand and engage end-users in technology adoption and use in context [11, 12, 13].”
BackgroundWang, J., Liang, H., Kang, H., & Gong, Y. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1678693 · Cited by 35
“25 Guided by the sociotechnical model, a composite classi fi cation of health ITrelated contributing factors was created and further used to investigate the health IT related sentinel events reported to the Joint Commission.”
MethodInterfacing With the Electronic Health Record (EHR): A Comparative Review of Modes of Documentation
Avendano, J. P., Gallagher, D. O., Hawes, J., Boyle, J. J., Glasser, L., Aryee, J. N. A., & Katt, B. M. (2022) · Cureus · doi:10.7759/cureus.26330 · Cited by 31
“While EHRs in their current form may improve the quality of clinical documentation, they frequently result in increased documentation times, decreased clinician satisfaction, decreased length of patient interaction, new sources of risk to patient safety, and substantial investment costs for providers [4,22,23].”
BackgroundEHR-BERT: A BERT-based model for effective anomaly detection in electronic health records
Niu, H., Omitaomu, O. A., Langston, M. A., Olama, M., Özmen, Ö., Klasky, H. B., Laurio, A., Ward, M. C., & Nebeker, J. R. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104605 · Cited by 44
Wright, A. T., Aaron, S., & Sittig, D. F. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw039 · Cited by 28
Self-citationManagement of Open Innovation in Healthcare for Cost Accounting Using EHR
Biancone, P., Secinaro, S. F., Brescia, V., & Calandra, D. (2019) · Journal of Open Innovation Technology Market and Complexity · doi:10.3390/joitmc5040099 · Cited by 30
The impact of electronic records on patient safety: a qualitative study
Scantlebury, A., Adamson, J., Watt, I. S., Sheard, L., Cairns, P. A., & Wright, J. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0299-y · Cited by 28
“S evidence base, which has identified human errors and technical issues associated with Health Information Technology [8–12, 27–33] and which have the potential to increase the risks for patient safety.”
BackgroundScheitel, M. R., Kessler, M., Shellum, J. L., Peters, S. G., Milliner, D. S., Liu, H., Elayavilli, R. K., Poterack, K. A., Miksch, T. A., Boysen, J. J., et al. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2016-07-ra-0114 · Cited by 28
Patient safety goals for the proposed Federal Health Information Technology Safety Center
Sittig, D. F., Classen, D. C., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002988 · Cited by 28
“For example, ONC could partner with not for profit entities (eg, ECRI’s recently formed ‘Partnership for promoting HIT patient safety’(18)) to create a federally funded research and development center for event reporting, analysis, and information sharing, similar in concept to the Veterans Affairs’ Informatics Patien…”
BackgroundSelf-citationSavoy, A., Militello, L. G., Patel, H., Flanagan, M., Russ, A. L., Daggy, J., Weiner, M., & Saleem, J. J. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.07.021 · Cited by 27
Ferreira, A. M. D., de Oliveira, J. L. C., Camillo, N. R. S., et al. (2019) · Revista gaúcha de enfermagem · doi:10.1590/1983-1447.2019.20180140 · Cited by 25
Human-Computer Interaction, Ethics, and Biomedical Informatics
Hochheiser, H., & Valdez, R. S. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1701990 · Cited by 26
“Consistent with earlier efforts [6-9], examinations of patient safety reports continue to find evidence of a strong impact of usability problems on EHR safety [10, 11].”
ResultRatwani, R. M., Hettinger, A. Z., Kosydar, A., Fairbanks, R. J. T., & Hodgkins, M. L. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw092 · Cited by 24
“Many EHRs are not prospectively designed, developed, and implemented to support the cognitive needs of clinicians, resulting in stress, frustration, reduced efficiency, and patient safety hazards.(4,5) In an effort to promote EHR usability, the Office of the National Coordinator for Health Information Technology (ONC)…”
BackgroundElectronic Health Records as an Educational Tool: Viewpoint
Habboush, Y. Y., Hoyt, R., & Beidas, S. (2018) · JMIR Medical Education · doi:10.2196/10306 · Cited by 22
“Other interpersonal and communication skills benefits from the huddle include promoting team dynamics and interdisciplinary work [20].”
Backgroundde Vreede, M. A. V., McGrath, A., & de Clifford, J. (2018) · Australian Health Review · doi:10.1071/ah17119 · Cited by 23
Health Information Technology Acceptance Framework for diabetes management
Ayanlade, O. S., Oyebisi, T. O., & Kolawole, B. A. (2019) · Heliyon · doi:10.1016/j.heliyon.2019.e01735 · Cited by 23
“That is, no matter be the usefulness of the technology, if it is not easy to understand and easy to interact with, then its usefulness is useless as con fi rmed by Meeks et al. (2014), as this will determine the HIT acceptance and sustainability among staff.”
BackgroundResultBednorz, A., Mak, J. K. L., Jylhävä, J., & Religa, D. (2023) · Clinical Interventions in Aging · doi:10.2147/cia.s400887 · Cited by 27
Kemp, T., Ayton, J. E., Butler-Henderson, K. A., & Lam, M. K. (2024) · Social Theory & Health · doi:10.1057/s41285-024-00214-5 · Cited by 37
“Meeks et al. reported that electronic health records have complex sociotechnical origins which require organisations to monitor and learn from system implementations and build robust infrastructure [27].”
Horsky, J., & Ramelson, H. Z. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.10.004 · Cited by 21
Engaging the disability community in informatics research: rationales and practical steps
Valdez, R. S., Lyon, S. E., Wellbeloved-Stone, C. A., Collins, M., Rogers, C. C., Cantin-Garside, K. D., Fortes, D., Kim, C., Desai, S. S., Keim‐Malpass, J., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac136 · Cited by 25
Clinical Information Systems – From Yesterday to Tomorrow
Gardner, R. M. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s010 · Cited by 22
“In addition, the treatment strategies of the medical profession are not completely interchangeable with each patient’s condition and are not performed without error [99, 100, 107-115].”
BackgroundEthics, Medicine, and Information Technology
Goodman, K. W. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781139600330 · Cited by 40
Ahlness, E. A., Orlander, J. D., Brunner, J., Cutrona, S. L., Kim, B., Molloy-Paolillo, B. K., Rinne, S. T., Rucci, J. M., Sayre, G. G., & Anderson, E. (2023) · Journal of General Internal Medicine · doi:10.1007/s11606-023-08282-5 · Cited by 21
“20 Furthermore, the emotional effects of disrupted work can impact employee confidence and morale, leading to burnout and turnover.”
BackgroundAmosa, T. I., Izhar, L. I., Sebastian, P., Ismail, I. B., Ibrahim, O., & Ayinla, S. L. (2023) · IEEE Access · doi:10.1109/access.2023.3284682 · Cited by 24
“To address the challenges of EHR-related patient safety, comprehensive and more recent ‘‘socio-technical’’ methods that account for these components are required [69], [70], [71].”
BackgroundMethodAdvocating for Greater Usability in Clinical Technologies
Lopez, K. D., & Fahey, L. J. (2018) · Critical Care Nursing Clinics of North America · doi:10.1016/j.cnc.2018.02.007 · Cited by 16
Nursing Student Experiences Regarding Safe Use of Electronic Health Records
Whitt, K. J., Eden, L. M., Merrill, K. C., et al. (2016) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000291 · Cited by 17
Kang, H., Wang, J., Yao, B., Zhou, S., & Gong, Y. (2018) · JAMIA Open · doi:10.1093/jamiaopen/ooy042 · Cited by 17
Martin, G., Arora, S., Shah, N., King, D., & Darzi, A. W. (2019) · Health Informatics Journal · doi:10.1177/1460458219854602 · Cited by 15
Khairat, S. S., Coleman, C., Teal, R., et al. (2021) · Health Informatics Journal · doi:10.1177/1460458221997914 · Cited by 14
Bae, J., Rask, K. J., & Becker, E. R. (2017) · American Journal of Medical Quality · doi:10.1177/1062860617702453 · Cited by 17
“Furthermore, it is not uncommon to find evidence that EMRs may lead to unintended consequences such as fostering new, unique safety risks and increasing the incidence of adverse patient safety events and medical errors.(24-26) The potential benefits accruing to an EMR investment are dependent in part on how well the v…”
BackgroundSafe use of hospital information systems: an evaluation model based on a sociotechnical perspective
Salahuddin, L. B., Ismail, Z., Ikram, R. R. R., Hashim, U. R., Idris, A., Ismail, N. H., Hassan, N. H., & Rahim, F. A. (2019) · Behaviour and Information Technology · doi:10.1080/0144929x.2019.1597164 · Cited by 14
“The root cause of HIS-related errors are complex, and originate from various technical and/or sociotechnical factors (Meeks et al. 2014; Vanderhook and Abraham 2017).”
BackgroundApplying requisite imagination to safeguard electronic health record transitions
Sittig, D. F., Lakhani, P., & Singh, H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab291 · Cited by 16
Self-citationBramo, S. S., Desta, A., & Syedda, M. (2022) · Digital Health · doi:10.1177/20552076221131144 · Cited by 18
“Consistently, studies conducted by 30,48 – 50 demonstrated simplicity to navigate and ease of access, clarity, and ef fi ciency as a factor associated with technology acceptance and use.”
BackgroundAssessing the Status Quo of EHR Accessibility, Usability, and Knowledge Dissemination
Khairat, S., Coleman, C., Russomagno, S., & Gotz, D. (2018) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.228 · Cited by 16
“More recently, safety concerns have been linked to EHRs in particular, specifically as they relate to workflow and data display [14].”
BackgroundLaboratory tests as short-term correlates of stroke
Sughrue, T., Swiernik, M. A., Huang, Y. H., & Brody, J. P. (2016) · BMC Neurology · doi:10.1186/s12883-016-0619-y · Cited by 13
“Electronic health records (EHRs) are rapidly being adopted by medical providers, and are now used by the majority of office-based physicians and hospitals in the US [6, 7].”
BackgroundElectronic health record competency in graduate nurses: A grounded theory study
Weinschreider, J., Tenzek, K. E., Foltz-Ramos, K., et al. (2023) · Nurse Education Today · doi:10.1016/j.nedt.2023.105987 · Cited by 13
Saurin, T. A., Patriarca, R., Hegde, S., & Rayo, M. F. (2024) · Applied Ergonomics · doi:10.1016/j.apergo.2024.104290 · Cited by 14
Fadahunsi, K. P., Wark, P. A., Mastellos, N., Neves, A. L., Gallagher, J., Majeed, A., Webster, A. P., Smith, A., Choo‐Kang, B., Leon, C., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/41889 · Cited by 12
“However, the quality of information obtained from DHTs varies and can compromise quality and safety of care [2-4].”
A Shared Opportunity for Improving Electronic Medical Record Data
Baier, A. W., Snyder, D., Leahy, I. C., Patak, L. S., & Brustowicz, R. M. (2017) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000002134 · Cited by 11
Analysis of variations in the display of drug names in computerized prescriber-order-entry systems
Quist, A., Hickman, T.-T. T., Amato, M. G., et al. (2017) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp151051 · Cited by 11
Modeling the mind: How do we design effective decision-support?
Weir, C. R., Rubin, M. A., Nebeker, J. R., & Samore, M. H. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.06.008 · Cited by 12
A Sociotechnical Approach to Electronic Health Record Related Safety
Sittig, D. F., & Singh, H. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809523-2.00014-5 · Cited by 11
Self-citationCampione, J. R., Mardon, R. E., & McDonald, K. M. (2018) · Journal of Patient Safety · doi:10.1097/pts.0000000000000531 · Cited by 10
Heed, J., Klein, S., Slee, A., Watson, N., Husband, A., & Slight, S. P. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15284 · Cited by 11
“Overuse of alerts or inappropriate alerting results in alert fatigue and a high incidence of alert overrides, which can adversely affect patient safety.(11) Van de Sijs et al.”
BackgroundHave Electronic Health Records Improved the Quality of Patient Care?
Krenn, L., & Schlossman, D. (2017) · PM&R · doi:10.1016/j.pmrj.2017.04.001 · Cited by 8
“A majority of providers cite remote access as one of the key benefits of having an EHR [7].”
BackgroundStakeholder perspectives of system-related errors: Types, contributing factors, and consequences
Kinlay, M., Zheng, W. Y., Burke, R. A., Juraskova, I., Ho, L. M., Turton, H., Trinh, J., & Baysari, M. T. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104821 · Cited by 9
Ong, M., Sylvia Pelayo, A., & Issues, S. E. F. T. I. Y. S. O. H. F. A. O. (2015) · Yearbook of Medical Informatics · doi:10.15265/iy-2015-033 · Cited by 10
“Two papers were already selected in two other sections [20-21], and were removed from the HFOI selection, because considered to be better relevant to these sections.”
BackgroundHealth information systems´ usability-related use errors in patient safety incidents
Hautamäki, E., Kinnunen, U., & Palojoki, S. (2017) · Finnish Journal of eHealth and eWelfare · doi:10.23996/fjhw.60763 · Cited by 10
“units and date formats), wording and functions [5,6], patient identification issues [6,7,9], navigation difficulties [6,7], and wrong or missing data input [7,910].”
BackgroundInfluentialA hospital-wide transition from paper to digital problem-oriented clinical notes
Cillessen, F., Robbé, P. D. V., & Biermans, M. C. J. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2016-08-ra-0137 · Cited by 11
Despins, L. A., & Wakefield, B. J. (2018) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000431 · Cited by 10
“Others have observed poor alignment of the EMR with assessment workflows, noting an inadequate fit between information needs and the task to be performed.(30,46)”
BackgroundReal-Time Automated Hazard Detection Framework for Health Information Technology Systems
Omitaomu, O. A., Özmen, Ö., Olama, M. M., Pullum, L., Kuruganti, T. P., Nutaro, J. J., Klasky, H. B., Zandi, H., Advani, A. A., Laurio, A., et al. (2019) · Health Systems · doi:10.1080/20476965.2019.1599701 · Cited by 11
“These concerns cover hazards that pose immense threats to patient safety as reported in multiple studies (e.g., Graber, Johnston, & Bailey, 2016; Magrabi, Ong, Runciman, & Coiera, 2011; Meeks et al., 2014; Wright et al., 2016).”
BackgroundSelf-citationAll CLEAR? Preparing for IT Downtime
Kashiwagi, D. T., Sexton, M. D., Graves, C., Johnson, J. M., Callies, B. I., Yu, R. C., & Thompson, J. M. (2016) · American Journal of Medical Quality · doi:10.1177/1062860616667546 · Cited by 9
Firdaus, M. (2019) · Jurnal Administrasi Rumah Sakit Indonesia · doi:10.7454/arsi.v6i1.2880 · Cited by 9
Metadata
- Authors
- Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H.
- Affiliations
- Michael E. DeBakey VA Medical Center (US)
- United States Department of Veterans Affairs (US)
- Memorial Hermann (US)
- Corresponding author
- Singh, Hardeep
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-11
- Volume
- 21
- Issue
- 6
- Pages
- 1053-1059
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC4215044
- OpenAlex
- W2148595420
- Semantic Scholar
- 9b75d5d6ec4e36813fe145af295d58b9926d0372
- Cited by
- 207
- References
- 60
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 16 · 2023: 13 · 2022: 14 · 2021: 15 · 2020: 10 · 2019: 24 · 2018: 22 · 2017: 25 · 2016: 30 · 2015: 20 · 2014: 2
- FWCI
- 46.82
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- U.S. Department of Veterans Affairs (14-274, CIN13-413, USA 14-274)
- Office of Research and Development (CIN 13-413)
- Health Services Research and Development (CIN 13‐413, CRE12-033, USA 14-274)
- Agency for Healthcare Research and Quality (R01HS022087)
- U.S. Public Health Service (T32HP10031)
- Center for Innovations in Quality, Effectiveness and Safety (CIN 13–413, 13–413)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- electronic health records, patient safety, socio-technical model, near misses, software upgrade, data transmission, hidden dependencies, workflow, EHR implementation, safety reporting, Veterans Health Administration
- MeSH terms
- Humans, Retrospective Studies, United States, United States Department of Veterans Affairs, Medical Errors, Workflow, Electronic Health Records, Patient Safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
