CoChat

Automated identification of extreme-risk events in clinical incident reports

Open access
Ong, Mei-Sing, Magrabi, Farah, Coiera, Enrico · 2012 · Journal of the American Medical Informatics Association
Cited by 45

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

OBJECTIVES: To explore the feasibility of using statistical text classification to automatically detect extreme-risk events in clinical incident reports. METHODS: Statistical text classifiers based on Naïve Bayes and Support Vector Machine (SVM) algorithms were trained and tested on clinical incident reports to automatically detect extreme-risk events, defined by incidents that satisfy the criteria of Severity Assessment Code (SAC) level 1. For this purpose, incident reports submitted to the Advanced Incident Management System by public hospitals from one Australian region were used. The classifiers were evaluated on two datasets: (1) a set of reports with diverse incident types (n=120); (2) a set of reports associated with patient misidentification (n=166). Results were assessed using accuracy, precision, recall, F-measure, and area under the curve (AUC) of receiver operating characteristic curves. RESULTS: The classifiers performed well on both datasets. In the multi-type dataset, SVM with a linear kernel performed best, identifying 85.8% of SAC level 1 incidents (precision=0.88, recall=0.83, F-measure=0.86, AUC=0.92). In the patient misidentification dataset, 96.4% of SAC level 1 incidents were detected when SVM with linear, polynomial or radial-basis function kernel was used (precision=0.99, recall=0.94, F-measure=0.96, AUC=0.98). Naïve Bayes showed reasonable performance, detecting 80.8% of SAC level 1 incidents in the multi-type dataset and 89.8% of SAC level 1 patient misidentification incidents. Overall, higher prediction accuracy was attained on the specialized dataset, compared with the multi-type dataset. CONCLUSION: Text classification techniques can be applied effectively to automate the detection of extreme-risk events in clinical incident reports.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/amiajnl-2011-000562

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 25

  1. 1999, TO err is Human: Building a safer Health System. Committee on Quality of Health Care in America, 86
  2. 2003, Evaluation of two methods for quality improvement in intensive care: facilitated incident monitoring and retrospective medical chart review, Crit Care Med, 31, 1006, 10.1097/01.CCM.0000060016.21525.3C · DOI · Paper page
  3. 2001, Developing and deploying a patient safety program in a large health care delivery system: you can't fix what you don't know about, Jt Comm J Qual Improv, 27, 522
  4. 1999, Reasons for not reporting adverse incidents: an empirical study, J Eval Clin Pract, 5, 13, 10.1046/j.1365-2753.1999.00147.x · DOI · Paper page
  5. 2009, Medication errors: how reliable are the severity ratings reported to the national reporting and learning system?, Int J Qual Health Care, 21, 316, 10.1093/intqhc/mzp034 · DOI
  6. University Centre for Clinical Governance Research in Health, Evaluation of the safety improvement program in New South Wales: Study no 6 report on program outcomes
  7. 2010, Multiple accountabilities in incident reporting and management, Qual Health Res, 20, 1091, 10.1177/1049732310369232 · DOI
  8. 2007, Automatic discovery of anomalies reported in aerospace systems health and safety documents
  9. 2010, Automated detection of harm in healthcare with information technology: a systematic review, Qual Saf Health Care, 19, e11
  10. 2007, Automatic identification and classification of surgical margin status from pathology reports following prostate cancer surgery, AMIA Annu Symp Proc, 11, 160
  11. 2003, Creating a text classifier to detect radiology reports describing mediastinal findings associated with inhalation anthrax and other disorders, J Am Med Inform Assoc, 10, 494, 10.1197/jamia.M1330 · DOI
  12. 2002, Use of natural language processing to translate clinical information from a database of 889,921 chest radiographic reports, Radiology, 224, 157, 10.1148/radiol.2241011118 · DOI · Paper page
  13. 2003, Detecting adverse events in discharge summaries using variations on the simple bayes model, AMIA Annu Symp Proc, 689
  14. 2005, Automated detection of adverse events using natural language processing of discharge summaries, J Am Med Inform Assoc, 12, 448, 10.1197/jamia.M1794 · DOI · Paper page
  15. 2011, Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection, J Am Med Inform Assoc, 18, 631, 10.1136/amiajnl-2010-000022 · DOI
  16. 2010, Automated categorisation of clinical incident reports using statistical text classification, Qual Saf Health Care, 19, e55
  17. 1988, The Australian patient safety Foundation, Anaesth Intensive Care, 16, 114, 10.1177/0310057X8801600139 · DOI
  18. 2002, Machine learning in automated text categorization, ACM Computing Surveys, 34, 1, 10.1145/505282.505283 · DOI · Paper page
  19. 1998, Text categorization with support vector machines: learning with many relevant features. Machine Learning: ECML-98, Lect Notes Comput Sci, 1398, 137, 10.1007/BFb0026683 · DOI · Paper page
  20. 2004, Applying support vector machines to imbalanced datasets, Lect Notes Comput Sci, 3201, 39, 10.1007/978-3-540-30115-8_7 · DOI · Paper page
  21. 2005, Text mining: Predictive Methods for Analyzing Unstructured Information, 30
  22. 1998, A study using n-gram features for text categorization
  23. 2001, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, 17
  24. 2004, Statistical Design for Research, 113
  25. 1999, The NASA Aviation Safety Reporting System: lessons learned from voluntary incident reporting, 97

Cited by

See all 45

Metadata

Authors
Ong, M.-S., Magrabi, F., & Coiera, E.
Venue
Journal of the American Medical Informatics Association
Published
2012-01-11
Volume
19
Issue
e1
Pages
e110-e118
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W2263224247
Cited by
45
References
3
FWCI
3.17
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Automated identification of extreme-risk events in clinical incident reports. https://app.cochat.ai/papers/10.1136/amiajnl-2011-000562
The paper
Ong, M.-S., Magrabi, F., & Coiera, E. (2012). Automated identification of extreme-risk events in clinical incident reports. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2011-000562

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.