Automated identification of extreme-risk events in clinical incident reports
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
OBJECTIVES: To explore the feasibility of using statistical text classification to automatically detect extreme-risk events in clinical incident reports. METHODS: Statistical text classifiers based on Naïve Bayes and Support Vector Machine (SVM) algorithms were trained and tested on clinical incident reports to automatically detect extreme-risk events, defined by incidents that satisfy the criteria of Severity Assessment Code (SAC) level 1. For this purpose, incident reports submitted to the Advanced Incident Management System by public hospitals from one Australian region were used. The classifiers were evaluated on two datasets: (1) a set of reports with diverse incident types (n=120); (2) a set of reports associated with patient misidentification (n=166). Results were assessed using accuracy, precision, recall, F-measure, and area under the curve (AUC) of receiver operating characteristic curves. RESULTS: The classifiers performed well on both datasets. In the multi-type dataset, SVM with a linear kernel performed best, identifying 85.8% of SAC level 1 incidents (precision=0.88, recall=0.83, F-measure=0.86, AUC=0.92). In the patient misidentification dataset, 96.4% of SAC level 1 incidents were detected when SVM with linear, polynomial or radial-basis function kernel was used (precision=0.99, recall=0.94, F-measure=0.96, AUC=0.98). Naïve Bayes showed reasonable performance, detecting 80.8% of SAC level 1 incidents in the multi-type dataset and 89.8% of SAC level 1 patient misidentification incidents. Overall, higher prediction accuracy was attained on the specialized dataset, compared with the multi-type dataset. CONCLUSION: Text classification techniques can be applied effectively to automate the detection of extreme-risk events in clinical incident reports.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Towards an International Classification for Patient Safety: key concepts and terms
Runciman, W., Hibbert, P., Thomson, R., Van Der Schaaf, T., Sherman, H., & Lewalle, P. (2009) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn057 · Cited by 647 · cited 1×
“An HIT is a descriptive term for a category made up of incidents of a common nature, grouped because of shared, agreed features.(23) Each HIT contains further sub-categories.”
BackgroundPhysician Reporting Compared with Medical-Record Review to Identify Adverse Medical Events
O'Neil, A. C., Petersen, L. A., Cook, E. F., Bates, D. W., Lee, T. H., & Brennan, T. A. (1993) · Annals of Internal Medicine · doi:10.7326/0003-4819-119-5-199309010-00004 · Cited by 315
Runciman, W. B. (2002) · Quality and Safety in Health Care · doi:10.1136/qhc.11.3.246 · Cited by 137
Research works printed but not linked: 25
- 1999, TO err is Human: Building a safer Health System. Committee on Quality of Health Care in America, 86
- 2003, Evaluation of two methods for quality improvement in intensive care: facilitated incident monitoring and retrospective medical chart review, Crit Care Med, 31, 1006, 10.1097/01.CCM.0000060016.21525.3C · DOI · Paper page
- 2001, Developing and deploying a patient safety program in a large health care delivery system: you can't fix what you don't know about, Jt Comm J Qual Improv, 27, 522
- 1999, Reasons for not reporting adverse incidents: an empirical study, J Eval Clin Pract, 5, 13, 10.1046/j.1365-2753.1999.00147.x · DOI · Paper page
- 2009, Medication errors: how reliable are the severity ratings reported to the national reporting and learning system?, Int J Qual Health Care, 21, 316, 10.1093/intqhc/mzp034 · DOI
- University Centre for Clinical Governance Research in Health, Evaluation of the safety improvement program in New South Wales: Study no 6 report on program outcomes
- 2010, Multiple accountabilities in incident reporting and management, Qual Health Res, 20, 1091, 10.1177/1049732310369232 · DOI
- 2007, Automatic discovery of anomalies reported in aerospace systems health and safety documents
- 2010, Automated detection of harm in healthcare with information technology: a systematic review, Qual Saf Health Care, 19, e11
- 2007, Automatic identification and classification of surgical margin status from pathology reports following prostate cancer surgery, AMIA Annu Symp Proc, 11, 160
- 2003, Creating a text classifier to detect radiology reports describing mediastinal findings associated with inhalation anthrax and other disorders, J Am Med Inform Assoc, 10, 494, 10.1197/jamia.M1330 · DOI
- 2002, Use of natural language processing to translate clinical information from a database of 889,921 chest radiographic reports, Radiology, 224, 157, 10.1148/radiol.2241011118 · DOI · Paper page
- 2003, Detecting adverse events in discharge summaries using variations on the simple bayes model, AMIA Annu Symp Proc, 689
- 2005, Automated detection of adverse events using natural language processing of discharge summaries, J Am Med Inform Assoc, 12, 448, 10.1197/jamia.M1794 · DOI · Paper page
- 2011, Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection, J Am Med Inform Assoc, 18, 631, 10.1136/amiajnl-2010-000022 · DOI
- 2010, Automated categorisation of clinical incident reports using statistical text classification, Qual Saf Health Care, 19, e55
- 1988, The Australian patient safety Foundation, Anaesth Intensive Care, 16, 114, 10.1177/0310057X8801600139 · DOI
- 2002, Machine learning in automated text categorization, ACM Computing Surveys, 34, 1, 10.1145/505282.505283 · DOI · Paper page
- 1998, Text categorization with support vector machines: learning with many relevant features. Machine Learning: ECML-98, Lect Notes Comput Sci, 1398, 137, 10.1007/BFb0026683 · DOI · Paper page
- 2004, Applying support vector machines to imbalanced datasets, Lect Notes Comput Sci, 3201, 39, 10.1007/978-3-540-30115-8_7 · DOI · Paper page
- 2005, Text mining: Predictive Methods for Analyzing Unstructured Information, 30
- 1998, A study using n-gram features for text categorization
- 2001, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, 17
- 2004, Statistical Design for Research, 113
- 1999, The NASA Aviation Safety Reporting System: lessons learned from voluntary incident reporting, 97
Cited by
Machine learning in medicine: a practical introduction
Sidey-Gibbons, J. A. M., & Sidey-Gibbons, C. J. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0681-4 · Cited by 1,271
Role of artificial intelligence in patient safety outcomes: systematic literature review
Choudhury, A., & Asan, O. (2020) · JMIR Medical Informatics · doi:10.2196/18599 · Cited by 440
De Micco, F., Di Palma, G., Ferorelli, D., De Benedictis, A., Tomassini, L., Tambone, V., Cingolani, M., & Scendoni, R. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1522554 · Cited by 58
Clinical research informatics: a conceptual perspective
Kahn, M. G., & Weng, C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-000968 · Cited by 40
Wang, H.-H., Huang, C.-C., Talley, P. C., & Kuo, K.-M. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/3100618 · Cited by 6
“Second, most studies focus on assessing the types of patient safety incidents [12, 13, 15, 16, 23, 26, 29].”
BackgroundMethodInfluential
Metadata
- Authors
- Ong, M.-S., Magrabi, F., & Coiera, E.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-01-11
- Volume
- 19
- Issue
- e1
- Pages
- e110-e118
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2263224247
- Cited by
- 45
- References
- 3
- FWCI
- 3.17
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automated identification of extreme-risk events in clinical incident reports. https://app.cochat.ai/papers/10.1136/amiajnl-2011-000562
- The paper
- Ong, M.-S., Magrabi, F., & Coiera, E. (2012). Automated identification of extreme-risk events in clinical incident reports. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2011-000562
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.