The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; none of the 34 sources its claims cite is checked for support yet.
An analysis of computer-related patient safety incidents to inform the development of a classification
Open accessAbstract
OBJECTIVE: To analyze patient safety incidents associated with computer use to develop the basis for a classification of problems reported by health professionals. DESIGN: Incidents submitted to a voluntary incident reporting database across one Australian state were retrieved and a subset (25%) was analyzed to identify 'natural categories' for classification. Two coders independently classified the remaining incidents into one or more categories. Free text descriptions were analyzed to identify contributing factors. Where available medical specialty, time of day and consequences were examined. MEASUREMENTS: Descriptive statistics; inter-rater reliability. RESULTS: A search of 42,616 incidents from 2003 to 2005 yielded 123 computer related incidents. After removing duplicate and unrelated incidents, 99 incidents describing 117 problems remained. A classification with 32 types of computer use problems was developed. Problems were grouped into information input (31%), transfer (20%), output (20%) and general technical (24%). Overall, 55% of problems were machine related and 45% were attributed to human-computer interaction. Delays in initiating and completing clinical tasks were a major consequence of machine related problems (70%) whereas rework was a major consequence of human-computer interaction problems (78%). While 38% (n=26) of the incidents were reported to have a noticeable consequence but no harm, 34% (n=23) had no noticeable consequence. CONCLUSION: Only 0.2% of all incidents reported were computer related. Further work is required to expand our classification using incident reports and other sources of information about healthcare IT problems. Evidence based user interface design must focus on the safe entry and retrieval of clinical information and support users in detecting and correcting errors and malfunctions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
49 Own result · 27 Not checked yet · 6 Own method · 6 Own interpretation · 5 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A search of 42 616 incidents from 2003 to 2005 yielded 123 computer related incidents.”
- Also stated in: § RESULTS · plain
“Our search strategy retrieved 123 incidents, 23 of which were retrieved using AIMS and the remainder from free-text searches of incident descriptions (see online supplementary appendix B, available at www.jamia.org).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
- Also stated in: § RESULTS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A classification with 32 types of computer use problems was developed.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After removing duplicate and unrelated incidents, 99 incidents describing 117 problems remained.”
- Also stated in: § Classification of computer-related problems · plain
“Examination of the remaining 99 incidents revealed 117 problems.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
- Also stated in: § Classification of computer-related problems · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“rework was a major consequence of humancomputer interaction problems (78%)”
- Also stated in: § Classification of computer-related problems · plain
“In contrast, rework was a major consequence of problems in humanecomputer interaction (78%, n¼18).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § Classification of computer-related problems · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Delays in initiating and completing clinical tasks were a major consequence of machine related problems (70%)”
- Also stated in: § Classification of computer-related problems · plain
“Delays in initiating and completing clinical tasks directly related to patient care were a major consequence of machine-related problems (70%, n¼39).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § Classification of computer-related problems · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Only 0.2% of all incidents reported were computer related.”
- Also stated in: § CONCLUSION · plain
“Only 0.2% of all incidents reported were computer related.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
- Also stated in: § CONCLUSION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Problems were grouped into information input (31%), transfer (20%), output (20%) and general technical (24%).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“General technical problems accounted for 24% of the incidents (n¼28).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > General technical
plainn = 28- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Use of hybrid papereelectronic systems sometimes resulted in omission errors where clinicians were not notified about results (eg, 'doctor/hospital team not notified about abnormal results from private abdominal ultrasound scan available on computer system, results available online but not sent to doctor as requested').”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Failed output devices prevented access to results.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 15
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, 55% of problems were machine related and 45% were attributed to human-computer interaction.”
- Also stated in: § Classification of computer-related problems · plain
“Of these, 55% (n¼64) were machine-related problems and 45% (n¼53) were problems in humanecomputer interaction (table 2).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § Classification of computer-related problems · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While 38% (n¼26) of the incidents were reported to have a noticeable consequence but no harm, 34% (n¼23) had no noticeable consequence.”
- Also stated in: § RESULTS · plain
“While 38% (n¼26) of the incidents were reported to have a noticeable consequence but no harm, 34% (n¼23) had no noticeable consequence.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Abstract
plainn = 26- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
- Also stated in: § RESULTS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Information output problems accounted for 20% of incidents (n¼23) caused by malfunctioning peripheral devices (n¼13, 11%; eg, printers and monitors).”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plaindirectn = 23- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some incidents (6.0%, n¼7) related to problems in updating data (eg, 'computer system not updated when patient transferred between wards', 'medication lists not updated on admission') and missing data (6.0%,n¼7) (eg, 'details of primary care physician not entered in online system').”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information input problems
plainn = 7- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Softwarerelated errors were detected by vigilant staff and required rework to correct mistakes (eg, 'letters needed to be re-done').”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > General technical
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Examination of these incidents (n¼8) revealed that the availability and sharing of electronic health records (EHR) aided detection of drugedrug interactions (eg, warfarine tramadol), duplicate medication orders and MRSA infection risk, assisted checking and correction of dialysis treatment, found discrepancies in antenatal paper records, and provided up-to-date contact details.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Improved patient safety
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have expanded these categories to identify specific manifestations of these problems and added two general categories to account for technical problems and contributing factors described by reporters.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Comparison with the literature
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As with network problems, poor performance or failure of a single workstation prevented staff from carrying out tasks (eg, 'nurses unable to access results and complete handovers', 'clinician unable to access care plan for frequently presenting patient who required treatment within 30 min in Emergency'), caused delays and were reported to be a source of significant frustration.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > General technical
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notification problems delayed treatment and were reported to be a source of frustration for staff who needed to act upon test results.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Information output problems
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Potential and actual breaches of patient privacy were reported to be the consequence of printouts left at patients' bedsides as well as failure to log-off the computer system.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Contributing factors
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Lack of access to the computer system (eg, EHR) often resulted in workarounds relying on paper-based records, which were often inefficient and ineffective.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Improving the safety of clinical IT
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that technical issues relating to computer hardware, software, or networking infrastructure accounted for over half the problems reported (55%), with human factors reported to be the primary cause in the remaining 45%.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Factors contributing to incidents
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the 111 incidents, eight described an improvement in patient safety due to IT, and four were unresolvable.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Classification of computer-related problems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Factors reported including the lack of training, failure to carry out a duty, high cognitive workload, and the effects of multitasking.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Factors contributing to incidents
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Delays in initiating or completing clinical tasks were reported to be a major consequence of the computer-related incidents we examined, and were associated with 70% of machine-related problems.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Consequences of incidents
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Six out of 10 problems in humane computer interaction related to data entry (64%), and retrieval of clinical and patient data was also problematic.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Factors contributing to incidents
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Machine-related problems (software-and hardware-related) accounted for more than half of the problems, with most of the remainder attributed to problems with humanecomputer interactions.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Fewer mismatches between actual clinical workflow and the system model were reported in comparison to Koppel et al's mixed method study, most likely representing a narrower focus on the part of clinicians forwarding incidents.8”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Comparison with the literature
What the sources say
- Not checked yetcites [8] source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Patient safety risks and incidents caused by problems with the use of computers are now widely recognized as an unintended consequence of healthcare information technology (IT), 1e3 just as 'revenge effects' have been described after other system changes.4”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites 1e3
- Not checked yetcites [4] source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“In February 2010, the US Food and Drug Administration (FDA) reported receiving information on 260 incidents with potential for patient harm including 44 injuries and six deaths.6”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [6] US Office of the National Coordinator for Health IT, HIT Policy Committee, Adoption/Certification Workgroup meeting 2010. http://healthit.hhs.gov/.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“However, with large collections of incidents, characteristic profiles may be identified, allowing incidents to be aggregated and analyzed.23”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [23] Symposium. The Australian Incident Monitoring System. Anaesth Intensive Care 1993;21:501–695. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“It is important to note that incident reports do not yield true frequencies of errors or adverse events because they do not capture numerators or denominators, and are subject to bias from a number of sources.22”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [22] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The framework is based upon existing classifications such as AIMS, with additional input from international experts in safety, systems engineering, health policy, medicine, and the law.5 28 29”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Two of our main categories map to the high-level error classes first identified by Ash et al. 1 'Information input problems' and 'information output problems' correspond to errors in 'entering and retrieving information', and 'information transfer problems' correspond to 'communication and co-ordination'.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Comparison with the literature
What the sources say
- Not checked yetcites [1] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Identifying the natural categories of safety incidents has been the method used for creating the AIMS classification, which is the starting point for expanding the ICPS.18 30”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Learning from incidents
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“When coders disagreed on a primary classification, the event was re-examined and a consensus category assigned.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Classification development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such a classification will allow information about computer-related incidents to be collated and classified, providing an objective basis for comparing patterns over time and between settings, and for the development and prioritization of preventive and corrective strategies.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Indeed, incident reporting to facilitate rapid communication of safety flaws and critical events arising from computer use is one of seven steps which have been proposed to improve the safety of healthcare IT. 20”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [20] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This was done without reference to AIMS to avoid constraining the range of new categories.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Classification development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Sittig and Classen 21 endorse the reporting of computer-related incidents as an essential component of their framework for safe use of IT systems.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [21] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Computers are listed as an option under the 'medical equipment/device' category within an equipment list that is sourced from the ECRI's Universal Medical Device Nomenclature System (UMDNS).26”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [26] Emergency Care Research Institute (ECRI) [Internet] 2010. http://www.ecri.org (accessed Nov 2008).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the system used (AIMS) does not specifically elicit information about IT incidents, which may have inhibited reporting (most incidents in our analysis were retrieved from free-text descriptions)”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Reporting and analyses of IT safety incidents
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The classification developed for AIMS allows incidents to be grouped according to 13 healthcare incident types (HIT), such as 'clinical process/procedure' or 'medication/ IV fluid.' 23e25”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites 23e25
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The Advanced Incident Management System (AIMS) 18 is one such incident reporting system, based on 20 years of research in patient safety, and has been in use since 1998 in more than 1000 facilities in Australia, New Zealand, South Africa, and the United States.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [18] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Recently, a framework for an International Classification for Patient Safety (ICPS) has been agreed to, by a drafting group convened by the World Health Organization (WHO) World Alliance for Patient Safety.27”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
What the sources say
- Not checked yetcites [27] International Classification for Patient Safety [Internet] 2010. http://www.who.int/patientsafety/en/ (accessed Feb 2010).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Another limitation is that the dataset used, from one Australian state, has limitations imposed by the education provided and practices which evolved in that state with respect to incident reporting.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The incidents studied here are based on self-reports provided to a voluntary incident reporting system, with all the inherent limitations of such a system, such as a bias toward reporting incidents which appear interesting or unusual.22”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
What the sources say
- Not checked yetcites [22] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As we have shown in other domains, such incident reports are useful in providing a profile of the nature of the problems encountered, and this profile has generally been shown to be consistent until interventions are introduced to address problems identified.22”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
What the sources say
- Not checked yetcites [22] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The computer-related problems we have identified are limited to the types of IT systems in use at the time and represent only a small proportion of the kinds of problems that might be encountered.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the incidents analyzed were reported over a significant period, providing sufficient data for some quantitative and qualitative analyses.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > Limitations of this dataset
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
The keyword-based free-text search strategy comprehensively identifies computer-related patient safety incidents in the AIMS database.
“Free-text searches of incident description fields were conducted using a set of keywords generated by the investigators to describe computer hardware, software, or displays based upon knowledge of the clinical information systems deployed in the jurisdiction (box 1).”
The paper relies on this without support.
The 25% subset of incidents used to develop natural categories is representative of the full set of computer-related incidents.
“We examined the free-text descriptions of a quarter of the incidents retrieved (25% of 123) to identify 'natural categories' for classification.30”
The paper relies on this without support.
This is the first study to examine computer-related patient safety incidents reported by health professionals to a state-wide system to develop categories for a classification.
“This is the first study we are aware of which examines computer-related patient safety incidents reported by health professionals to a state-wide system in order to develop categories to provide the basis for a classification.”
The paper relies on this without support.
Patient safety incidents associated with computer use have not been extensively investigated.
“Patient safety incidents associated with computer use have not been extensively investigated.”
The paper relies on this without support.
The types of error are finite in number and, when systematically identified and addressed, can lead to improvements in patient safety.
“the types of error are finite in number and, when systematically identified and addressed, can lead to improvements in patient safety.”
The paper relies on this without support.
Incident reports are useful for learning even when no actual harm resulted, when clinicians feel that there has been a near miss or that a catastrophic outcome could have resulted.
“Incident reports are useful for learning even when no actual harm resulted, when clinicians feel that there has been a near miss or that a catastrophic outcome could have resulted.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A few studies have attempted to quantify and classify EHR-related safety concerns by mining patient safety incident reporting databases.18 29–31”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [30]
In the citing paper
“Previous studies have largely used isolated event reports without benefit of an independent human factors and informatics investigation to analyze safety concerns in the EHR.”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ An analysis of computer-related patient safety incidents to inform the development of a classification > INTRODUCTION
“A similar categorization was used by Koppel et al 8 to describe the causes of 22 types of medication error specifically associated with computerized physician order entry (CPOE) systems at one US hospital.”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > References
“8. Ash JS, Sittig DF, Dykstra R, et al. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform 2009;78(Suppl 1):S69e76.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| 4.2 Software not available | 1 (2) | | |”
§ An analysis of computer-related patient safety incidents to inform the development of a classification > CONCLUSION
Reported 2 · recomputed 0.854701
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789
Han, Y. Y., Carcillo, J. A., Venkataraman, S. T., Clark, R. S. B., Watson, R. S., Nguyen, T. C., Bayir, H., & Orr, R. A. (2005) · Pediatrics · doi:10.1542/peds.2005-1287 · Cited by 1,149
Types of Unintended Consequences Related to Computerized Provider Order Entry
Campbell, E. M., Sittig, D. F., Ash, J. S., Guappone, K. P., & Dykstra, R. H. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2042 · Cited by 823
Ammenwerth, E., Schnell‐Inderst, P., Machan, C., & Siebert, U. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2667 · Cited by 681
Towards an International Classification for Patient Safety: key concepts and terms
Runciman, W., Hibbert, P., Thomson, R., Van Der Schaaf, T., Sherman, H., & Lewalle, P. (2009) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn057 · Cited by 647
Self-citationHigh Rates of Adverse Drug Events in a Highly Computerized Hospital
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.10.1111 · Cited by 363
Adverse drug events and medication errors in Australia
Runciman, W. B. (2003) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzg085 · Cited by 355
Self-citation
Research works printed but not linked: 1
- Australian Medication Safety Working Group. Towards the safer use of drugs and blood products in Australia. J Qual Clin Pract 1999;19:1–72.
Cited by
Sorted by citations
Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., Payne, T. H., Rosenbloom, S. T., Weaver, C., & Zhang, J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001458 · Cited by 618
Health IT and Patient Safety: Building Safer Systems for Better Care
Chou, D. (2012) · JAMA · doi:10.1001/jama.308.21.2282-a · Cited by 496
Horsky, J., Schiff, G. D., Johnston, D., Mercincavage, L. M., Bell, D. S., & Middleton, B. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.09.002 · Cited by 307
Kim, M. O., Coiera, E., & Magrabi, F. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw154 · Cited by 263
“For example, users ignored alerts (n1⁄46) and failed to update information.(13,14) Many delays in clinical decisionmaking were linked to computer network issues.”
BackgroundInfluentialSelf-citationAn analysis of electronic health record-related patient safety concerns
Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002578 · Cited by 207
Metadata
- Authors
- Magrabi, F., Ong, M.-S., Runciman, W., & Coiera, E.
- Affiliations
- UNSW Sydney (AU)
- University of South Australia (AU)
- Corresponding author
- Magrabi, Farah
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-11
- Volume
- 17
- Issue
- 6
- Pages
- 663-670
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3000751
- OpenAlex
- W2127433315
- Semantic Scholar
- bdc378bbf6e6f3103cce3c62c6eadd155032f3d8
- Cited by
- 162
- References
- 35
- Citations by year
- 2026: 8 · 2025: 3 · 2024: 10 · 2023: 9 · 2022: 8 · 2021: 8 · 2020: 4 · 2019: 14 · 2018: 7 · 2017: 10 · 2016: 14 · 2015: 24 · 2014: 8 · 2013: 17 · 2012: 12
- FWCI
- 16.58
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- University of New South Wales
- Australian Research Council (DP0772487, LP0775532)
- National Health and Medical Research Council (568612)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- incident classification, human-computer interaction, health information technology, medical information retrieval, user interface design, incident reporting, medical errors, rework, inter-rater reliability
- MeSH terms
- Australia, Computer Systems, Humans, Medical Informatics, Risk Management, User-Computer Interface, Information Storage and Retrieval, Medical Errors, Equipment Failure Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
