Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Learning from patient safety incident reports is a vital part of improving healthcare. However, the volume of reports and their largely free-text nature poses a major analytic challenge. The objective of this study was to test the capability of autonomous classifying of free text within patient safety incident reports to determine incident type and the severity of harm outcome. Primary care patient safety incident reports (n=31333) previously expert-categorised by clinicians (training data) were processed using J48, SVM and Naïve Bayes. The SVM classifier was the highest scoring classifier for incident type (AUROC, 0.891) and severity of harm (AUROC, 0.708). Incident reports containing deaths were most easily classified, correctly identifying 72.82% of reports. In conclusion, supervised ML can be used to classify patient safety incident report categories. The severity classifier, whilst not accurate enough to replace manual processing, could provide a valuable screening tool for this critical aspect of patient safety.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
Medical error—the third leading cause of death in the US
Makary, M. A., & Daniel, M. (2016) · BMJ · doi:10.1136/bmj.i2139 · Cited by 3,264
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712
Kimia, A. A., Savova, G., Landschaft, A., & Harper, M. B. (2015) · Pediatric Emergency Care · doi:10.1097/pec.0000000000000484 · Cited by 103
Cited by
Role of artificial intelligence in patient safety outcomes: systematic literature review
Choudhury, A., & Asan, O. (2020) · JMIR Medical Informatics · doi:10.2196/18599 · Cited by 440
Threat Detection and Response Using AI and NLP in Cybersecurity
Ismail, D. W. S. (2024) · Journal of Internet Services and Information Security · doi:10.58346/jisis.2024.i1.013 · Cited by 34
“Security incidents can be less harmful when automated actions, including limiting malicious activity or quarantining compromised devices, are carried out quickly (Evans HP, 2020).”
Wang, H.-H., Huang, C.-C., Talley, P. C., & Kuo, K.-M. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/3100618 · Cited by 6
“Second, most studies focus on assessing the types of patient safety incidents [12, 13, 15, 16, 23, 26, 29].”
BackgroundInfluential
Metadata
- Authors
- Evans, H. P., Anastasiou, A., Edwards, A., Hibbert, P., Makeham, M., Luz, S., Sheikh, A., Donaldson, L., & Carson-Stevens, A.
- Venue
- Health Informatics Journal
- Publisher
- SAGE Publications
- Published
- 2019-03-07
- Volume
- 26
- Issue
- 4
- Pages
- 3123-3139
- ISSN
- 1355-4743 (ISSN-L)
- 1460-4582
- 1741-2811
- OpenAlex
- W2922262511
- Cited by
- 47
- References
- 4
- FWCI
- 2.19
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches. https://app.cochat.ai/papers/10.1177/1460458219833102
- The paper
- Evans, H. P., Anastasiou, A., Edwards, A., Hibbert, P., Makeham, M., Luz, S., Sheikh, A., Donaldson, L., & Carson-Stevens, A. (2020). Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches. Health Informatics Journal. https://doi.org/10.1177/1460458219833102
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.