The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; none of the 34 sources its claims cite is checked for support yet.
An analysis of computer-related patient safety incidents to inform the development of a classification
Open accessAbstract
OBJECTIVE: To analyze patient safety incidents associated with computer use to develop the basis for a classification of problems reported by health professionals. DESIGN: Incidents submitted to a voluntary incident reporting database across one Australian state were retrieved and a subset (25%) was analyzed to identify 'natural categories' for classification. Two coders independently classified the remaining incidents into one or more categories. Free text descriptions were analyzed to identify contributing factors. Where available medical specialty, time of day and consequences were examined. MEASUREMENTS: Descriptive statistics; inter-rater reliability. RESULTS: A search of 42,616 incidents from 2003 to 2005 yielded 123 computer related incidents. After removing duplicate and unrelated incidents, 99 incidents describing 117 problems remained. A classification with 32 types of computer use problems was developed. Problems were grouped into information input (31%), transfer (20%), output (20%) and general technical (24%). Overall, 55% of problems were machine related and 45% were attributed to human-computer interaction. Delays in initiating and completing clinical tasks were a major consequence of machine related problems (70%) whereas rework was a major consequence of human-computer interaction problems (78%). While 38% (n=26) of the incidents were reported to have a noticeable consequence but no harm, 34% (n=23) had no noticeable consequence. CONCLUSION: Only 0.2% of all incidents reported were computer related. Further work is required to expand our classification using incident reports and other sources of information about healthcare IT problems. Evidence based user interface design must focus on the safe entry and retrieval of clinical information and support users in detecting and correcting errors and malfunctions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
49 Own result · 27 Not checked yet · 6 Own method · 6 Own interpretation · 5 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A search of 42 616 incidents from 2003 to 2005 yielded 123 computer related incidents.”
- Also stated in: § RESULTS · plain
“Our search strategy retrieved 123 incidents, 23 of which were retrieved using AIMS and the remainder from free-text searches of incident descriptions (see online supplementary appendix B, available at www.jamia.org).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
- Also stated in: § RESULTS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A classification with 32 types of computer use problems was developed.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After removing duplicate and unrelated incidents, 99 incidents describing 117 problems remained.”
- Also stated in: § Classification of computer-related problems · plain
“Examination of the remaining 99 incidents revealed 117 problems.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
- Also stated in: § Classification of computer-related problems · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“rework was a major consequence of humancomputer interaction problems (78%)”
- Also stated in: § Classification of computer-related problems · plain
“In contrast, rework was a major consequence of problems in humanecomputer interaction (78%, n¼18).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § Classification of computer-related problems · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Delays in initiating and completing clinical tasks were a major consequence of machine related problems (70%)”
- Also stated in: § Classification of computer-related problems · plain
“Delays in initiating and completing clinical tasks directly related to patient care were a major consequence of machine-related problems (70%, n¼39).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § Classification of computer-related problems · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Only 0.2% of all incidents reported were computer related.”
- Also stated in: § CONCLUSION · plain
“Only 0.2% of all incidents reported were computer related.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
- Also stated in: § CONCLUSION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Problems were grouped into information input (31%), transfer (20%), output (20%) and general technical (24%).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“General technical problems accounted for 24% of the incidents (n¼28).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > General technical
plainn = 28- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Use of hybrid papereelectronic systems sometimes resulted in omission errors where clinicians were not notified about results (eg, 'doctor/hospital team not notified about abnormal results from private abdominal ultrasound scan available on computer system, results available online but not sent to doctor as requested').”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Failed output devices prevented access to results.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 15
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, 55% of problems were machine related and 45% were attributed to human-computer interaction.”
- Also stated in: § Classification of computer-related problems · plain
“Of these, 55% (n¼64) were machine-related problems and 45% (n¼53) were problems in humanecomputer interaction (table 2).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § Classification of computer-related problems · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While 38% (n¼26) of the incidents were reported to have a noticeable consequence but no harm, 34% (n¼23) had no noticeable consequence.”
- Also stated in: § RESULTS · plain
“While 38% (n¼26) of the incidents were reported to have a noticeable consequence but no harm, 34% (n¼23) had no noticeable consequence.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plainn = 26- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § RESULTS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Information output problems accounted for 20% of incidents (n¼23) caused by malfunctioning peripheral devices (n¼13, 11%; eg, printers and monitors).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plaindirectn = 23- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some incidents (6.0%, n¼7) related to problems in updating data (eg, 'computer system not updated when patient transferred between wards', 'medication lists not updated on admission') and missing data (6.0%,n¼7) (eg, 'details of primary care physician not entered in online system').”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information input problems
plainn = 7- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Softwarerelated errors were detected by vigilant staff and required rework to correct mistakes (eg, 'letters needed to be re-done').”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > General technical
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Examination of these incidents (n¼8) revealed that the availability and sharing of electronic health records (EHR) aided detection of drugedrug interactions (eg, warfarine tramadol), duplicate medication orders and MRSA infection risk, assisted checking and correction of dialysis treatment, found discrepancies in antenatal paper records, and provided up-to-date contact details.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Improved patient safety
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have expanded these categories to identify specific manifestations of these problems and added two general categories to account for technical problems and contributing factors described by reporters.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Comparison with the literature
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As with network problems, poor performance or failure of a single workstation prevented staff from carrying out tasks (eg, 'nurses unable to access results and complete handovers', 'clinician unable to access care plan for frequently presenting patient who required treatment within 30 min in Emergency'), caused delays and were reported to be a source of significant frustration.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > General technical
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notification problems delayed treatment and were reported to be a source of frustration for staff who needed to act upon test results.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Potential and actual breaches of patient privacy were reported to be the consequence of printouts left at patients' bedsides as well as failure to log-off the computer system.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Contributing factors
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Lack of access to the computer system (eg, EHR) often resulted in workarounds relying on paper-based records, which were often inefficient and ineffective.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Improving the safety of clinical IT
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that technical issues relating to computer hardware, software, or networking infrastructure accounted for over half the problems reported (55%), with human factors reported to be the primary cause in the remaining 45%.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Factors contributing to incidents
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the 111 incidents, eight described an improvement in patient safety due to IT, and four were unresolvable.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Classification of computer-related problems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Factors reported including the lack of training, failure to carry out a duty, high cognitive workload, and the effects of multitasking.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Factors contributing to incidents
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Delays in initiating or completing clinical tasks were reported to be a major consequence of the computer-related incidents we examined, and were associated with 70% of machine-related problems.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Consequences of incidents
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Six out of 10 problems in humane computer interaction related to data entry (64%), and retrieval of clinical and patient data was also problematic.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Factors contributing to incidents
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Machine-related problems (software-and hardware-related) accounted for more than half of the problems, with most of the remainder attributed to problems with humanecomputer interactions.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Fewer mismatches between actual clinical workflow and the system model were reported in comparison to Koppel et al's mixed method study, most likely representing a narrower focus on the part of clinicians forwarding incidents.8”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Comparison with the literature
What the sources say
- Not checked yetcites [8] source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Patient safety risks and incidents caused by problems with the use of computers are now widely recognized as an unintended consequence of healthcare information technology (IT), 1e3 just as 'revenge effects' have been described after other system changes.4”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites 1e3
- Not checked yetcites [4] source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“In February 2010, the US Food and Drug Administration (FDA) reported receiving information on 260 incidents with potential for patient harm including 44 injuries and six deaths.6”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [6] US Office of the National Coordinator for Health IT, HIT Policy Committee, Adoption/Certification Workgroup meeting 2010. http://healthit.hhs.gov/.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“However, with large collections of incidents, characteristic profiles may be identified, allowing incidents to be aggregated and analyzed.23”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [23] Symposium. The Australian Incident Monitoring System. Anaesth Intensive Care 1993;21:501–695. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“It is important to note that incident reports do not yield true frequencies of errors or adverse events because they do not capture numerators or denominators, and are subject to bias from a number of sources.22”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [22] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The framework is based upon existing classifications such as AIMS, with additional input from international experts in safety, systems engineering, health policy, medicine, and the law.5 28 29”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Two of our main categories map to the high-level error classes first identified by Ash et al. 1 'Information input problems' and 'information output problems' correspond to errors in 'entering and retrieving information', and 'information transfer problems' correspond to 'communication and co-ordination'.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Comparison with the literature
What the sources say
- Not checked yetcites [1] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Identifying the natural categories of safety incidents has been the method used for creating the AIMS classification, which is the starting point for expanding the ICPS.18 30”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Learning from incidents
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“When coders disagreed on a primary classification, the event was re-examined and a consensus category assigned.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Classification development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such a classification will allow information about computer-related incidents to be collated and classified, providing an objective basis for comparing patterns over time and between settings, and for the development and prioritization of preventive and corrective strategies.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Indeed, incident reporting to facilitate rapid communication of safety flaws and critical events arising from computer use is one of seven steps which have been proposed to improve the safety of healthcare IT. 20”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [20] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This was done without reference to AIMS to avoid constraining the range of new categories.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Classification development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Sittig and Classen 21 endorse the reporting of computer-related incidents as an essential component of their framework for safe use of IT systems.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [21] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Computers are listed as an option under the 'medical equipment/device' category within an equipment list that is sourced from the ECRI's Universal Medical Device Nomenclature System (UMDNS).26”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [26] Emergency Care Research Institute (ECRI) [Internet] 2010. http://www.ecri.org (accessed Nov 2008).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the system used (AIMS) does not specifically elicit information about IT incidents, which may have inhibited reporting (most incidents in our analysis were retrieved from free-text descriptions)”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Reporting and analyses of IT safety incidents
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The classification developed for AIMS allows incidents to be grouped according to 13 healthcare incident types (HIT), such as 'clinical process/procedure' or 'medication/ IV fluid.' 23e25”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites 23e25
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The Advanced Incident Management System (AIMS) 18 is one such incident reporting system, based on 20 years of research in patient safety, and has been in use since 1998 in more than 1000 facilities in Australia, New Zealand, South Africa, and the United States.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [18] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Recently, a framework for an International Classification for Patient Safety (ICPS) has been agreed to, by a drafting group convened by the World Health Organization (WHO) World Alliance for Patient Safety.27”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [27] International Classification for Patient Safety [Internet] 2010. http://www.who.int/patientsafety/en/ (accessed Feb 2010).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Another limitation is that the dataset used, from one Australian state, has limitations imposed by the education provided and practices which evolved in that state with respect to incident reporting.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The incidents studied here are based on self-reports provided to a voluntary incident reporting system, with all the inherent limitations of such a system, such as a bias toward reporting incidents which appear interesting or unusual.22”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
What the sources say
- Not checked yetcites [22] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As we have shown in other domains, such incident reports are useful in providing a profile of the nature of the problems encountered, and this profile has generally been shown to be consistent until interventions are introduced to address problems identified.22”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
What the sources say
- Not checked yetcites [22] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The computer-related problems we have identified are limited to the types of IT systems in use at the time and represent only a small proportion of the kinds of problems that might be encountered.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the incidents analyzed were reported over a significant period, providing sufficient data for some quantitative and qualitative analyses.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
“Free-text searches of incident description fields were conducted using a set of keywords generated by the investigators to describe computer hardware, software, or displays based upon knowledge of the clinical information systems deployed in the jurisdiction (box 1).”
The paper relies on this without support.
The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
“We examined the free-text descriptions of a quarter of the incidents retrieved (25% of 123) to identify 'natural categories' for classification.30”
The paper relies on this without support.
This is the first study to examine computer-related patient safety incidents reported by health professionals to a state-wide system to develop categories for a classification.
“This is the first study we are aware of which examines computer-related patient safety incidents reported by health professionals to a state-wide system in order to develop categories to provide the basis for a classification.”
The paper relies on this without support.
Patient safety incidents associated with computer use have not been extensively investigated.
“Patient safety incidents associated with computer use have not been extensively investigated.”
The paper relies on this without support.
The types of error are finite in number and, when systematically identified and addressed, can lead to improvements in patient safety.
“the types of error are finite in number and, when systematically identified and addressed, can lead to improvements in patient safety.”
The paper relies on this without support.
Incident reports are useful for learning even when no actual harm resulted, when clinicians feel that there has been a near miss or that a catastrophic outcome could have resulted.
“Incident reports are useful for learning even when no actual harm resulted, when clinicians feel that there has been a near miss or that a catastrophic outcome could have resulted.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A few studies have attempted to quantify and classify EHR-related safety concerns by mining patient safety incident reporting databases.18 29–31”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [30]
In the citing paper
“Previous studies have largely used isolated event reports without benefit of an independent human factors and informatics investigation to analyze safety concerns in the EHR.”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
“A similar categorization was used by Koppel et al 8 to describe the causes of 22 types of medication error specifically associated with computerized physician order entry (CPOE) systems at one US hospital.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > References
“8. Ash JS, Sittig DF, Dykstra R, et al. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform 2009;78(Suppl 1):S69e76.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| 4.2 Software not available | 1 (2) | | |”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > CONCLUSION
Reported 2 · recomputed 0.854701
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789
Han, Y. Y., Carcillo, J. A., Venkataraman, S. T., Clark, R. S. B., Watson, R. S., Nguyen, T. C., Bayir, H., & Orr, R. A. (2005) · Pediatrics · doi:10.1542/peds.2005-1287 · Cited by 1,149
Types of Unintended Consequences Related to Computerized Provider Order Entry
Campbell, E. M., Sittig, D. F., Ash, J. S., Guappone, K. P., & Dykstra, R. H. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2042 · Cited by 823
Ammenwerth, E., Schnell‐Inderst, P., Machan, C., & Siebert, U. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2667 · Cited by 681
Towards an International Classification for Patient Safety: key concepts and terms
Runciman, W., Hibbert, P., Thomson, R., Van Der Schaaf, T., Sherman, H., & Lewalle, P. (2009) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn057 · Cited by 647
Self-citationHigh Rates of Adverse Drug Events in a Highly Computerized Hospital
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.10.1111 · Cited by 363
Adverse drug events and medication errors in Australia
Runciman, W. B. (2003) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzg085 · Cited by 355
Self-citation
Research works printed but not linked: 1
- Australian Medication Safety Working Group. Towards the safer use of drugs and blood products in Australia. J Qual Clin Pract 1999;19:1–72.
Cited by
Sorted by citations
Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., Payne, T. H., Rosenbloom, S. T., Weaver, C., & Zhang, J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001458 · Cited by 618
Health IT and Patient Safety: Building Safer Systems for Better Care
Chou, D. (2012) · JAMA · doi:10.1001/jama.308.21.2282-a · Cited by 496
Horsky, J., Schiff, G. D., Johnston, D., Mercincavage, L. M., Bell, D. S., & Middleton, B. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.09.002 · Cited by 307
Kim, M. O., Coiera, E., & Magrabi, F. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw154 · Cited by 263
“For example, users ignored alerts (n1⁄46) and failed to update information.(13,14) Many delays in clinical decisionmaking were linked to computer network issues.”
BackgroundInfluentialSelf-citationAn analysis of electronic health record-related patient safety concerns
Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002578 · Cited by 207
Using FDA reports to inform a classification for health information technology safety problems
Magrabi, F., Ong, M.-S., Runciman, W., & Coiera, E. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000369 · Cited by 193
Self-citationSingh, H., & Sittig, D. F. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004486 · Cited by 167
Identifying electronic health record usability and safety challenges in pediatric settings
Ratwani, R. M., Savage, E., Will, A., Fong, A., Karavite, D., Muthu, N., Rivera, A. J., Gibson, C., Asmonga, D., Moscovitch, B., et al. (2018) · Health Affairs · doi:10.1377/hlthaff.2018.0699 · Cited by 163
Howell, A.-M., Burns, E. M., Hull, L., Mayer, E. K., Sevdalis, N., & Darzi, A. W. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004456 · Cited by 131
An analysis of electronic health record–related patient safety incidents
Palojoki, S., Mäkelä, M., Lehtonen, L., & Saranto, K. (2016) · Health Informatics Journal · doi:10.1177/1460458216631072 · Cited by 120
The unintended consequences of health information technology revisited
Ash, J., Berg, M., & Coiera, E. (2016) · Yearbook of Medical Informatics · doi:10.15265/iy-2016-014 · Cited by 101
“…the most comprehensive attempt at integrating our knowledge of the different types is the classification scheme developed by Magrabi, Coiera, and colleagues, which utilizes patient safety concepts, and has been driven by analysis of incident reports from Australia, the US, and the UK [16, 18, 19].”
Self-citationCoiera, E. W., Aarts, J., & Kulikowski, C. A. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000674 · Cited by 107
“17 Differences in information models between systems means that semantic and technical interoperability always remains imperfect.”
BackgroundSelf-citationMagrabi, F., Baker, M., Sinha, I., Ong, M.-S., Harrison, S., Kidd, M. R., Runciman, W. B., & Coiera, E. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.12.003 · Cited by 100
Self-citationMeeks, D. W., Takian, A., Sittig, D. F., Singh, H., & Barber, N. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001762 · Cited by 91
Challenges to nurses' efforts of retrieving, documenting, and communicating patient care information
Keenan, G. M., Yakel, E., Lopez, K. D., Tschannen, D. J., & Ford, Y. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-000894 · Cited by 89
Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015) · JAMA · doi:10.1001/jama.2015.8372 · Cited by 91
Macroergonomics in Health Care Quality and Patient Safety
Carayon, P., Karsh, B., Gürses, A. P., Holden, R. J., Hoonakker, P., Hundt, A. S., Montague, E. N. H., Rodriguez, A. J., & Wetterneck, T. B. (2013) · Reviews of Human Factors and Ergonomics · doi:10.1177/1557234x13492976 · Cited by 93
Clinical decision support alert malfunctions: analysis and empirically derived taxonomy
Wright, A. T., Ai, A., Ash, J. S., Wiesen, J. F., Hickman, T.-T. T., Aaron, S., McEvoy, D. S., Borkowsky, S., Dissanayake, P. I., Embí, P. J., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx106 · Cited by 82
“42 Magrabi et al. 43 developed a 32-category taxonomy of computer patient safety incidents in Australia and subsequently expanded it to 36 categories by reviewing data from the US Food and Drug Administration.”
BackgroundA comparative review of patient safety initiatives for national health information technology
Magrabi, F., Aarts, J., Nøhr, C. G., Baker, M., Harrison, S., Sylvia Pelayo, A., Talmon, J. L., Sittig, D. F., & Coiera, E. W. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.11.014 · Cited by 74
Self-citationUsing statistical text classification to identify health information technology incidents
Chai, K. E. K., Anthony, S., Coiera, E. W., & Magrabi, F. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001409 · Cited by 68
Self-citationSalahuddin, L. B., & Ismail, Z. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.07.004 · Cited by 66
“missing results [54,64].”
BackgroundInfluentialMagrabi, F., Liaw, S. T., Arachi, D., Runciman, W. B., Coiera, E. W., & Kidd, M. R. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004323 · Cited by 60
“15 20 We estimated that a mail-out to 2000 GPs would be sufficient, allowing for a response rate of 10% (participation rates in online incident reporting had been 26%, 15 and 40% for a mail survey 6 ).”
BackgroundMethodInfluentialSelf-citationElectronic Health Records and Improved Patient Care: Opportunities for Applied Psychology
Ratwani, R. M. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721417700691 · Cited by 70
Classification of medication incidents associated with information technology
Cheung, K., van der Veen, W., Bouvy, M. L., Wensing, M., van den Bemt, P. M. L. A., & Smet, P. A. G. M. D. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001818 · Cited by 62
Improving the Effectiveness of Electronic Health Record-Based Referral Processes
Esquivel, A., Sittig, D. F., Murphy, D. R., & Singh, H. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-107 · Cited by 57
“Recent literature has revealed several serious health information technology-related errors that arose from faulty system design, configuration, or implementation processes [98-101].”
BackgroundKushniruk, A. W., Nøhr, C. G., Jensen, S., & Borycki, E. M. (2013) · PubMed · Cited by 54
Real-Time Heart Pulse Monitoring Technique Using Wireless Sensor Network and Mobile Application
Ali, N. S., Alyasseri, Z. A. A., & Abdulmohson, A. (2018) · International Journal of Electrical and Computer Engineering (IJECE) · doi:10.11591/ijece.v8i6.pp5118-5126 · Cited by 56
“Today, wireless technology has been eliminated all of the time-consuming jobs as well; the patients could be independent of instrumentation and bed [11-13].”
BackgroundLyell, D., Wang, Y., Coiera, E. W., & Magrabi, F. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad065 · Cited by 61
Self-citationSafety huddles to proactively identify and address electronic health record safety
Menon, S., Singh, H., Giardina, T. D., Rayburn, W. L., Davis, B. P., Russo, E. M., & Sittig, D. F. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw153 · Cited by 48
Wright, M. C., Borbolla, D. A., Waller, R. G., Fiol, G. D., Reese, T., Nesbitt, P., & Segall, N. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.yjbinx.2019.100041 · Cited by 48
“However, the ability of information displays to support these activities has been criticized [3–5], and their use has been linked with errors that compromise patient safety [6,7].”
BackgroundMartin, G., Ghafur, S., Cingolani, I., Symons, J., King, D., Arora, S., & Darzi, A. W. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30057-3 · Cited by 51
Coiera, E. W., & Liu, S. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100860 · Cited by 56
“Artificial intelligence safety It is now well understood that, along with many potential benefits, digital health can lead to patient harm if poorly designed, implemented, or used.(110) A review of the US FDA reports found 11% of IT-related incidents were associated with patient harm or death.”
BackgroundSelf-citationTechnology, cognition and error
Coiera, E. W. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003484 · Cited by 48
Self-citationHannaford, N., Mandel, C., Crock, C., Buckley, K. S., Magrabi, F., Ong, M., Allen, S., & Schultz, T. (2013) · British Journal of Radiology · doi:10.1259/bjr.20120336 · Cited by 49
Self-citationSafety Assurance Factors for Electronic Health Record Resilience (SAFER): study protocol
Singh, H., Ash, J. S., & Sittig, D. F. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-46 · Cited by 53
“The areas of expertise to be represented on the expert panel will include EHRs, patient safety, quality improvement, risk management, human factors engineering and usability, knowledge and experience with small physician office practices, and accreditation and certification practices for patient safety.”
BackgroundInfluentialMagrabi, F., Ong, M.-S., Runciman, W. B., & Coiera, E. W. (2011) · PubMed Central · Cited by 48
Self-citationEHR‐related medication errors in two ICUs
Carayon, P., Du, S., Brown, R., Cartmill, R. S., Johnson, M., & Wetterneck, T. B. (2017) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21259 · Cited by 43
The aspects of running artificial intelligence in emergency care; a scoping review
(2023) · Archives of Academic Emergency Medicine · doi:10.22037/aaem.v11i1.1974 · Cited by 45
Creating an Oversight Infrastructure for Electronic Health Record–Related Patient Safety Hazards
Singh, H., Classen, D. C., & Sittig, D. F. (2011) · Journal of Patient Safety · doi:10.1097/pts.0b013e31823d8df0 · Cited by 48
“DATA GATHERING OPERATIONS OF THE PROGRAM Voluntary reporting could continue to be an important source of data about EHR patient safety risks.(18) However, existing U.”
BackgroundRahman Jabin, M. S., & Pan, D. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231203600 · Cited by 41
“Six incidents were identi fi ed under this theme (IR 2,5,6,17,18,22 ).”
BackgroundKushniruk, A. W., Nøhr, C. G., Takeda, H., Kuwata, S., Carvalho, C., Bainbridge, M., Kannry, J. L., & Borycki, E. M. (2013) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638828 · Cited by 42
“Unlike many other more traditional software errors (found during software testing such as white or black-box testing), technolo-gy-induced errors are often unobservable until systems are used under real-world conditions and/or in real-world settings (e.g. hospitals, clinics) [5, 14, 17].”
BackgroundJabin, M. S. R., Pan, D., & Nilsson, E. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1260521 · Cited by 41
Edwards, P., & Warm, D. (2012) · Applied Clinical Informatics · doi:10.4338/aci-2012-03-ra-0010 · Cited by 38
“The use of a framework such as that suggested by Magrabi et al [10, 11] is important in helping clinicians understand the types of incidents relating to the ever increasing use of health information technologies.”
MethodInfluentialUsability Flaws in Medication Alerting Systems: Impact on Usage and Work System
Ammenwerth, E., Roehrer, E., Sylvia Pelayo, A., Vasseur, F., Beuscart-Zéphir, M. C., & Marcilly, R. (2015) · Yearbook of Medical Informatics · doi:10.15265/iy-2015-006 · Cited by 35
“usability [6].”
Rahman Jabin, M. S., & Hammar, T. (2022) · DIGITAL HEALTH · doi:10.1177/20552076221131139 · Cited by 32
“36,42 Using the keywords, 24 electronic prescription-related incidents were identi fi ed (see Table 1), which were then subjected to an existing framework, i.e., the HIT-CS.”
InfluentialPan, D., Nilsson, E., & Rahman Jabin, M. S. (2024) · Health Informatics Journal · doi:10.1177/14604582241270742 · Cited by 32
Adams, K., Howe, J. L., Fong, A., Puthumana, J. S., Kellogg, K., Gaunt, M. J., & Ratwani, R. M. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2017-01-ra-0014 · Cited by 32
“One study of patient safety hazards associated with computer use by healthcare providers reported that 20% of the safety hazard reports were associated with information transfer and about half of these were related to system integration problems [8].”
BackgroundWhat is the safety case for health IT? A study of assurance practices in England
Habli, I., White, S., Sujan, M. A., Harrison, S., & Ugarte, M. (2018) · Safety Science · doi:10.1016/j.ssci.2018.09.001 · Cited by 31
Fong, A., Adams, K. T., Gaunt, M. J., Howe, J. L., Kellogg, K. M., & Ratwani, R. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.09.007 · Cited by 32
Nohr, C., Jensen, S., Borycki, E. M., & Kushniruk, A. W. (2013) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638836 · Cited by 30
“Although many vendors of HIT now operate commercial usability laboratories where healthcare software systems products are designed and tested, reports of serious usability and safety issues continue to be reported globally [3-5,10].”
BackgroundInfluentialWang, J., Liang, H., Kang, H., & Gong, Y. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1678693 · Cited by 35
Borycki, E. M., Carayon, P., Jaspers, M., Sylvia Pelayo, A., & Beuscart-Zéphir, M.-C. (2013) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638835 · Cited by 32
“However recent research work on databases of patient safety incidents [27,28] demonstrated that: • The databases investigated do host reports of computer or HIT related incidents, but in very small proportion (0,1 % to 0,2%); this percentage is expected to grow in the future • A thorough secondary…”
BackgroundInfluentialA call for national e‐health clinical safety governance
Coiera, E. W., Kidd, M. R., & Haikerwal, M. C. (2012) · The Medical Journal of Australia · doi:10.5694/mja12.10475 · Cited by 28
Self-citationPatient safety goals for the proposed Federal Health Information Technology Safety Center
Sittig, D. F., Classen, D. C., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002988 · Cited by 28
Borycki, E. M. (2013) · Healthcare Informatics Research · doi:10.4258/hir.2013.19.2.69 · Cited by 26
“There is general agreement among researchers and policymakers that technology-induced errors arise from a number of sources [10-16,18-20].”
BackgroundInfluentialRights and responsibilities of users of electronic health records
Sittig, D. F., & Singh, H. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.111599 · Cited by 30
Measuring the effects of computer downtime on hospital pathology processes
Wang, Y., Coiera, E. W., Gallego, B., Perez‐Concha, O., Ong, M., Tsafnat, G., Roffe, D., Jones, G. R. D., & Magrabi, F. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.12.016 · Cited by 27
Self-citationEffect of an electronic medication administration record application on patient safety
Oliveros, N. V., Gramage, T., Menéndez-Conde, C. P., et al. (2017) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12753 · Cited by 27
A nationwide medication incidents reporting system in The Netherlands
Cheung, K., van den Bemt, P. M. L. A., Bouvy, M. L., et al. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000191 · Cited by 26
Human-Computer Interaction, Ethics, and Biomedical Informatics
Hochheiser, H., & Valdez, R. S. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1701990 · Cited by 26
“Consistent with earlier efforts [6-9], examinations of patient safety reports continue to find evidence of a strong impact of usability problems on EHR safety [10, 11].”
ResultCoiera, E. W. (2013) · The Medical Journal of Australia · doi:10.5694/mja13.10101 · Cited by 24
Self-citationWatbled, L., Marcilly, R., Guerlinger, S., Bastien, J. M. C., Beuscart-Zéphir, M.-C., & Beuscart, R. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.12.014 · Cited by 20
Carman, E.-M., Fray, M., & Waterson, P. E. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2020.103339 · Cited by 23
Syndromic surveillance for health information system failures: a feasibility study
Ong, M.-S., Magrabi, F., & Coiera, E. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001144 · Cited by 22
Self-citationSeth, M., Jalo, H., Högstedt, Å., Medin, O., Sjöqvist, B. A., & Candefjord, S. (2024) · Journal of Medical Internet Research · doi:10.2196/54470 · Cited by 29
“…within silos, potentially resulting in delayed information transfer, suboptimal decision-making, and patient harm [7,21,25,26]; for example, a study conducted by Magrabi et al [27] showed that approximately 20% of reported patient safety hazards were linked to deficient information transfer.”
Jabin, M. S. R., Magrabi, F., Hibbert, P. D., Schultz, T., & Runciman, W. B. (2019) · IEEE International Conference on Imaging Systems and Techniques (IST) · doi:10.1109/ist48021.2019.9010109 · Cited by 19
“The bulk of the keywords was drawn from the work of Farah Magrabi (FM) [17].”
BackgroundSelf-citationKinlay, M., Ho, L. M., Zheng, W. Y., Burke, R. A., Juraskova, I., Moles, R. J., & Baysari, M. T. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1739196 · Cited by 19
Vélez‐Díaz‐Pallarés, M., Álvarez‐Díaz, A. M., Gramage, T., Oliveros, N. V., Delgado‐Silveira, E., García, M. M., Cruz-Jentoft, A. J., & Bermejo-Vicedo, T. (2017) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-017-0474-y · Cited by 22
Marcilly, R., Schiro, J., Beuscart-Zéphir, M. C., & Magrabi, F. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1691841 · Cited by 20
Self-citationWyatt, K. D., Benning, T. J., Morgenthaler, T. I., & Arteaga, G. M. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1717084 · Cited by 16
Hyvämäki, P. J., Sneck, S., Meriläinen, M. H., Pikkarainen, M. A., Kääriäinen, M., & Jansson, M. M. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105045 · Cited by 17
Tabaie, A., Sengupta, S., Pruitt, Z. M., & Fong, A. (2023) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2022-100731 · Cited by 18
“17 Instead, these factors are often described in free-text narratives.”
Development of a Preliminary Patient Safety Classification System for Generative AI
Hose, B., Handley, J. L., Biro, J., Reddy, S., Krevat, S. A., Hettinger, A. Z., & Ratwani, R. M. (2025) · BMJ Quality & Safety · doi:10.1136/bmjqs-2024-017918 · Cited by 22
Past and Next 10 Years of Medical Informatics
Ückert, F., Ammenwerth, E., Dujat, C., et al. (2014) · Journal of Medical Systems · doi:10.1007/s10916-014-0074-5 · Cited by 17
“In the design of a CDSS, the interactions between patients, physicians, nurses, information systems and medical devices in the context of the tasksmust be considered holistically [3,7,8,14].”
BackgroundComparing two anesthesia information management system user interfaces: a usability evaluation
Wanderer, J. P., Rao, A. V., Rothwell, S., et al. (2012) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-012-9771-z · Cited by 16
Martin, G., Arora, S., Shah, N., King, D., & Darzi, A. W. (2019) · Health Informatics Journal · doi:10.1177/1460458219854602 · Cited by 15
“26 Using a deductive approach to analysis, the HITS Framework was employed to categorise statements across the three domains of safe health IT, using health IT safely and monitoring safety 24 with direct excerpts used as illustrative examples.”
BackgroundInfluentialMoghadam, S. T., Hooman, N., & Sheikhtaheri, A. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104150 · Cited by 16
Electronic approaches to making sense of the text in the adverse event reporting system
Benin, A. L., Fodeh, S. J., Lee, K., Koss, M., Miller, P. L., & Brandt, C. (2016) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21237 · Cited by 15
Health IT for Patient Safety and Improving the Safety of Health IT
Magrabi, F., Mei-Sing, O., & Coiera, E. W. (2016) · Studies in health technology and informatics · doi:10.3233/978-1-61499-635-4-25 · Cited by 16
“Errors can arise from unexpected interactions between system modules or with other systems [12].”
BackgroundInfluentialSelf-citationUnintended consequences of pharmacy information systems: A case study
Rohani, N., & Yusof, M. M. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104958 · Cited by 13
Zaouter, C., Wehbe, M., Cyr, S., et al. (2013) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-013-9466-1 · Cited by 13
“A recent study has indicated that 70 % of computerrelated medical incidents are caused by a delay in initiating a clinical procedure [18].”
BackgroundNguyen, L., Bakewell, L., Wickramasinghe, N. S., Haddad, P., Muhammad, I., Moghimi, H., Wilde, D., Redley, B., Considine, J., & Botti, M. (2015) · Cited by 13
Etablierung einer Telenotarzt-Anwendung
Kuntosch, J., Brinkrolf, P., Metelmann, C., Metelmann, B., Fischer, L., Hirsch, F., Süss, R., Fleßa, S., Kozlowski, T., Rübsam, M.-L., et al. (2020) · Springer eBooks · doi:10.1007/978-3-662-61930-8_4 · Cited by 9
Essential activities for electronic health record safety: A qualitative study
Ash, J. S., Singh, H., Wright, A., Chase, D., & Sittig, D. F. (2020) · Health Informatics Journal · doi:10.1177/1460458219833109 · Cited by 12
Hellot-Guersing, M., Jarre, C., Molina, C., Leromain, A., Derharoutunian, C., Gadot, A., & Roubille, R. (2015) · Annales Pharmaceutiques Françaises · doi:10.1016/j.pharma.2015.06.001 · Cited by 12
Borycki, E. M., Kushniruk, A. W., & Carvalho, C. (2013) · Computational and Mathematical Methods in Medicine · doi:10.1155/2013/526419 · Cited by 14
“These studies have found there to be instances where HISs have contributed to potential and real harms/deaths in patients [4, 7, 17].”
BackgroundMoghadam, S. T., Hooman, N., & Sheikhtaheri, A. (2022) · Studies in health technology and informatics · doi:10.3233/shti220362 · Cited by 12
Moghadam, S. T., Sheikhtaheri, A., & Hooman, N. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104549 · Cited by 11
Ong, M., Sylvia Pelayo, A., & Issues, S. E. F. T. I. Y. S. O. H. F. A. O. (2015) · Yearbook of Medical Informatics · doi:10.15265/iy-2015-033 · Cited by 10
“While health information technology (HIT) promises to enhance patient care quality, safety, and efficiency, significant concerns about its unintended consequences have been raised [1-3].”
BackgroundSelf-citationEffect of EHR User Interface Changes on Internal Prescription Discrepancies
Sawarkar, A., Dementieva, Y. A., Breydo, E. M., Ramelson, H. Z., & Turchin, A. (2014) · Applied Clinical Informatics · doi:10.4338/aci-2014-03-ra-0023 · Cited by 10
A SYSTEMATIC LITERATURE REVIEW ON SAFE HEALTH INFORMATION TECHNOLOGY USE BEHAVIOUR
Salahuddin, L. B., Ismail, Z., & Abidin, W. Z. (2016) · Jurnal Teknologi · doi:10.11113/jt.v78.8225 · Cited by 9
“Likewise, workarounds were the most common strategies to deal with technical problems which prevented access to patients’ and clinical information [27].”
BackgroundHabli, I., Jia, Y., White, S., Gabriel, G., Lawton, T., Sujan, M. A., & Tomsett, C. (2019) · Health Informatics Journal · doi:10.1177/1460458219852789 · Cited by 8
“15 In other words, there are many different ways in which HIT could fail to meet its intended purpose (e.g. due to usability, hardware, software and network errors).”
BackgroundUsing patient safety-event report data to assess health-IT safety: benefits and challenges
Howe, J. L., Hettinger, A. Z., & Ratwani, R. M. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30060-3 · Cited by 8
Kang, H., Yu, Z., & Gong, Y. (2017) · PubMed Central · Cited by 9
Noselli, M., van der Vegt, A. H., Cordeil, M., Campbell, V. K., Wang, A. P., Shetty, A., & Scott, I. A. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3572494 · Cited by 6
“Evaluation of UIs of AI-CDSS was often absent (7 systems) or of poor quality (8 systems) which hinders our ability to assess the comparative value of different UI designs which may, if deployed in clinical practice, result in poor adoption levels or potentially unsafe patient care [68], [81], [82].”
Kang, H., Wang, F., Zhou, S., Qi, M., & Gong, Y. (2017) · Studies in health technology and informatics · doi:10.3233/978-1-61499-830-3-1048 · Cited by 6
A Systems Model of HIT-Induced Complexity
Kuziemsky, C. E., & Ghazzawi, A. (2019) · Springer eBooks · doi:10.1007/978-3-030-10940-0_16 · Cited by 6
Jabin, M. S. R. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1786358 · Cited by 6
Safety Use of Hospital Information Systems: A Preliminary Investigation
Salahuddin, L. B., & Ismail, Z. (2015) · Lecture notes in business information processing · doi:10.1007/978-3-319-21009-4_52 · Cited by 5
Iqbal, A. R., Parau, C. A., Kazi, S., Adams, K., La, L., Hettinger, A. Z., & Ratwani, R. M. (2021) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/j.jcjq.2021.09.004 · Cited by 5
Metadata
- Authors
- Magrabi, F., Ong, M.-S., Runciman, W., & Coiera, E.
- Affiliations
- UNSW Sydney (AU)
- University of South Australia (AU)
- Corresponding author
- Magrabi, Farah
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-11
- Volume
- 17
- Issue
- 6
- Pages
- 663-670
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3000751
- OpenAlex
- W2127433315
- Semantic Scholar
- bdc378bbf6e6f3103cce3c62c6eadd155032f3d8
- Cited by
- 162
- References
- 35
- Citations by year
- 2026: 8 · 2025: 3 · 2024: 10 · 2023: 9 · 2022: 8 · 2021: 8 · 2020: 4 · 2019: 14 · 2018: 7 · 2017: 10 · 2016: 14 · 2015: 24 · 2014: 8 · 2013: 17 · 2012: 12
- FWCI
- 16.58
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- University of New South Wales
- Australian Research Council (DP0772487, LP0775532)
- National Health and Medical Research Council (568612)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- incident classification, human-computer interaction, health information technology, medical information retrieval, user interface design, incident reporting, medical errors, rework, inter-rater reliability
- MeSH terms
- Australia, Computer Systems, Humans, Medical Informatics, Risk Management, User-Computer Interface, Information Storage and Retrieval, Medical Errors, Equipment Failure Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
