A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
CONTEXT Adverse events in healthcare are often collated in incident reports which contain unstructured free text. Learning from these events may improve patient safety. Natural language processing (NLP) uses computational techniques to interrogate free text, reducing the human workload associated with its analysis. There is growing interest in applying NLP to patient safety, but the evidence in the field has not been summarised and evaluated to date. OBJECTIVE To perform a systematic literature review and narrative synthesis to describe and evaluate NLP methods for classification of incident reports and adverse events in healthcare. METHODS Data sources included Medline, Embase, The Cochrane Library, CINAHL, MIDIRS, ISI Web of Science, SciELO, Google Scholar, PROSPERO, hand searching of key articles, and OpenGrey. Data items were manually abstracted to a standardised extraction form. RESULTS From 428 articles screened for eligibility, 35 met the inclusion criteria of using NLP to perform a classification task on incident reports, or with the aim of detecting adverse events. The majority of studies used free text from incident reporting systems or electronic health records. Models were typically designed to classify by type of incident, type of medication error, or harm severity. A broad range of NLP techniques are demonstrated to perform these classification tasks with favourable performance outcomes. There are methodological challenges in how these results can be interpreted in a broader context. CONCLUSION NLP can generate meaningful information from unstructured data in the specific domain of the classification of incident reports and adverse events. Understanding what or why incidents are occurring is important in adverse event analysis. If NLP enables these insights to be drawn from larger datasets it may improve the learning from adverse events in healthcare.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Lawton, R., McEachan, R. R. C., Giles, S. J., Sirriyeh, R., Watt, I. S., & Wright, J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000443 · Cited by 311
Research works printed but not linked: 56
- Wood, 2005, Mandatory state-based error-reporting systems: current and future prospects, Am. J. Med. Qual., 20, 297, 10.1177/1062860605281850 · DOI
- Liang, 2017, Automated classification of multi-labeled patient safety reports: a shift from quantity to quality measure, Stud. Health Technol. Inform., 245, 1070
- Govindan, 2010, Automated detection of harm in healthcare with information technology: a systematic review, Qual. Saf. Health Care, 19, e11
- Carrell, 2017, Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings, J. Am. Med. Inform. Assoc., 24, 986, 10.1093/jamia/ocx039 · DOI · Paper page
- Jacobsson, 2017, Extraction of adverse event severity information from clinical narratives using natural language processing, Pharmacoepidemiol. Drug Saf., 26, 37
- Tian, 2017, Automated extraction of vte events from narrative radiology reports in electronic health records: a validation study, Med. Care, 55, e73, 10.1097/MLR.0000000000000346 · DOI
- Galvez, 2017, The use of natural language processing on pediatric diagnostic radiology reports in the electronic health record to identify deep venous thrombosis in children, J. Thromb. Thrombolysis, 44, 281, 10.1007/s11239-017-1532-y · doi:10.1007/s11239-017-1532-y
- Yim, 2017, Classifying tumor event attributes in radiology reports, J. Assoc. Inf. Sci. Technol., 68, 2662, 10.1002/asi.23937 · DOI
- Moore, 2017, Using natural language processing to extract abnormal results from cancer screening reports, J. Patient Saf., 13, 138, 10.1097/PTS.0000000000000127 · DOI
- Yetisgen-Yildiz, 2011, Automatic identification of critical follow-up recommendation sentences in radiology reports, AMIA Annu. Symp. Proc. 2011, 1593
- Duke, 2010, ADESSA: a real-time decision support service for delivery of semantically coded adverse drug event data, AMIA Annu. Symp. Proc. 2010, 177
- Li, 2015, Automated detection of medication administration errors in neonatal intensive care, J. Biomed. Inform., 57, 124, 10.1016/j.jbi.2015.07.012 · DOI
- Ni, 2018, Designing and evaluating an automated system for real-time medication administration error detection in a neonatal intensive care unit, J. Am. Med. Inform. Assoc., 25, 555, 10.1093/jamia/ocx156 · DOI
- Cai, 2016, Natural language processing to rapidly identify potential signals for adverse events using electronic medical record data: example of arthralgias and vedolizumab, Arthritis Rheumatol., 68, 2802
- Nunes, 2016, Assessing occurrence of hypoglycemia and its severity from electronic health records of patients with type 2 diabetes mellitus, Diabetes Res. Clin. Pract., 121, 192, 10.1016/j.diabres.2016.09.012 · DOI
- Toyabe, 2015, Characteristics of Inpatient Falls not Reported in an Incident Reporting System, Glob. J. Health Sci., 8, 17, 10.5539/gjhs.v8n3p17 · DOI
- Tanushi, 2014, Detection of healthcare-associated urinary tract infection in Swedish electronic health records, Stud. Health Technol. Inform., 207, 330
- Jensen, 2017, Analysis of free text in electronic health records for identification of cancer patient trajectories, Sci. Rep., 7, 46226, 10.1038/srep46226 · DOI · Paper page
- F. A, 2015, An evaluation of patient safety event report categories using unsupervised topic modeling, Methods Inf. Med., 54, 338, 10.3414/ME15-01-0010 · DOI
- Ong, 2010, Automated categorisation of clinical incident reports using statistical text classification, Qual. Saf. Health Care, 19, e55
- Gupta, 2015, Automated classification of clinical incident types, Stud. Health Technol. Inform., 214, 87
- Melton, 2005, Automated detection of adverse events using natural language processing of discharge summaries, J. Am. Med. Inform. Assoc., 12, 448, 10.1197/jamia.M1794 · DOI · Paper page
- Penz, 2007, Automated identification of adverse events related to central venous catheters, J. Biomed. Inform., 40, 174, 10.1016/j.jbi.2006.06.003 · DOI
- Ong, 2012, Automated identification of extreme-risk events in clinical incident reports, J. Am. Med. Inform. Assoc., 19, e110, 10.1136/amiajnl-2011-000562 · DOI · Paper page
- Murff, 2011, Automated identification of postoperative complications within an electronic medical record using natural language processing, JAMA, 306, 848
- Boyce, 2017, Automated screening of emergency department notes for drug-associated bleeding adverse events occurring in older adults, Appl. Clin. Inform., 8, 1022, 10.4338/ACI-2017-02-RA-0036 · doi:10.4338/aci-2017-02-ra-0036
- G. J, 2013, Automated validation of patient safety clinical incident classification: macro analysis, Stud. Health Technol. Inform., 188, 52
- Fujita, 2013, Detecting effective classes of medical incident reports based on linguistic analysis for common reporting system in Japan, Stud. Health Technol. Inform., 192, 137
- Toyabe, 2012, Detecting inpatient falls by using natural language processing of electronic medical records, BMC Health Serv. Res., 12, 448, 10.1186/1472-6963-12-448 · DOI
- Han, 2017, Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system, J. Am. Med. Inform. Assoc., 24, 913, 10.1093/jamia/ocx022 · DOI
- Benin, 2016, Electronic approaches to making sense of the text in the adverse event reporting system, J. Healthc. Risk Manag., 36, 10, 10.1002/jhrm.21237 · DOI
- Liang, 2015, Enhancing patient safety event reporting by K-nearest neighbor classifier, Stud. Health Technol. Inform., 218, 40603
- Fong, 2015, Exploring methods for identifying related patient safety events using structured and unstructured data, J. Biomed. Inform., 58, 89, 10.1016/j.jbi.2015.09.011 · DOI
- Speroff, 2013, Exploring the frontier of electronic health record surveillance the case of postoperative complications, Med. Care, 51, 509, 10.1097/MLR.0b013e31828d1210 · DOI · Paper page
- Gaebel, 2015, Extraction of adverse events from clinical documents to support decision making using semantic preprocessing, Stud. Health Technol. Inform., 216, 1030
- Iqbal, 2015, Identification of adverse drug events from free text electronic patient records and information in a large mental health case register, PLoS One, 10, e0134208, 10.1371/journal.pone.0134208 · DOI
- Fong, 2017, Integrating natural language processing expertise with patient safety event review committees to improve the analysis of medication events, Int. J. Med. Inform., 104, 120, 10.1016/j.ijmedinf.2017.05.005 · DOI
- Weller, 2017, Leveraging electronic health records for predictive modeling of post-surgical complications, Stat. Methods Med. Res., 1
- Fujita, 2012, Linguistic analysis of large-scale medical incident reports for patient safety, Stud. Health Technol. Inform., 180, 250
- Ravindranath, 2017, Machine learning in automated classification of adverse events in clinical studies of Alzheimer’s disease, Alzheimer’s Dement., 13, P1256, 10.1016/j.jalz.2017.06.1871 · doi:10.1016/j.jalz.2017.06.1871
- Liang, 2017, Predicting Harm Scores from Patient Safety Event Reports, Stud. Health Technol. Inform., 245, 1075
- W.M M, 2017, Screening electronic health record-related patient safety reports using machine learning, J. Patient Saf., 13, 31, 10.1097/PTS.0000000000000104 · DOI · Paper page
- McKnight, 2012, Semi-supervised classification of patient safety event reports, J. Patient Saf., 8, 60, 10.1097/PTS.0b013e31824ab987 · DOI
- Wong, 2016, Statistical classification of drug incidents due to look-alike sound-alike mix-ups, Health Informatics J., 22, 276, 10.1177/1460458214555040 · doi:10.1177/1460458214555040
- Wong, 2013, Statistical text classifier to detect specific type of medical incidents, Stud. Health Technol. Inform., 192, 1053
- Gerdes, 2013, Text mining electronic health records to identify hospital adverse events, Stud. Health Technol. Inform., 192, 1145
- Botsis, 2011, Text mining for the vaccine adverse event reporting system: medical text classification using informative feature selection, J. Am. Med. Inform. Assoc., 18, 631, 10.1136/amiajnl-2010-000022 · DOI
- Fong, 2017, Using active learning to identify health information technology related patient safety events, Appl. Clin. Inform., 8, 35
- Wang, 2017, Using multiclass classification to automate the identification of patient safety incident reports by type and severity, BMC Med. Inform. Decis. Mak., 17, 84, 10.1186/s12911-017-0483-8 · DOI · Paper page
- Botsis, 2012, Vaccine adverse event text mining system for extracting features from vaccine safety reports, J. Am. Med. Inform. Assoc., 19, 1011, 10.1136/amiajnl-2012-000881 · DOI
- Penz, 2007, Automated identification of adverse events related to central venous catheters, J. Biomed. Inform., 40, 174, 10.1016/j.jbi.2006.06.003 · DOI
- Stehman, 1997, Selecting and interpreting measures of thematic classification accuracy, Remote Sens. Environ., 62, 77, 10.1016/S0034-4257(97)00083-7 · DOI · Paper page
- Chubak, 2012, Tradeoffs between accuracy measures for electronic health care data algorithms, J. Clin. Epidemiol., 65, 343, 10.1016/j.jclinepi.2011.09.002 · DOI · Paper page
- Murdoch, 2013, The Inevitable Application of Big Data to Health Care, JAMA Evid., 309, 1351, 10.1001/jama.2013.393 · DOI · Paper page
- Adam, 2017, Data science, learning, and applications to biomedical and health sciences, Ann. N. Y. Acad. Sci., 1387, 5, 10.1111/nyas.13309 · DOI
- Young, 2018, Getting the measure of diabetes: the evolution of the national diabetes audit, Pract. Diabetes, 35, 1
Cited by
Role of artificial intelligence in patient safety outcomes: systematic literature review
Choudhury, A., & Asan, O. (2020) · JMIR Medical Informatics · doi:10.2196/18599 · Cited by 440
Reducing medication errors for adults in hospital settings
Ciapponi, A., Fernandez Nievas, S. E., Seijo, M., Rodríguez, M. B., Vietto, V., García-Perdomo, H. A., Virgilio, S., Fajreldines, A. V., Tost, J., Rose, C. J., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009985.pub2 · Cited by 105
Hah, H., & Goldin, D. S. (2021) · Journal of Medical Internet Research · doi:10.2196/33540 · Cited by 81
De Micco, F., Di Palma, G., Ferorelli, D., De Benedictis, A., Tomassini, L., Tambone, V., Cingolani, M., & Scendoni, R. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1522554 · Cited by 58
Yang, J., Wang, L., Phadke, N. A., Wickner, P. G., Mancini, C. M., Blumenthal, K. G., & Zhou, L. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.22836 · Cited by 42
Metadata
- Authors
- Young, I. J. B., Luz, S., & Lone, N.
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 132
- Pages
- 103971
- ISSN
- 1386-5056 (ISSN-L)
- OpenAlex
- W2978315017
- Cited by
- 139
- References
- 2
- FWCI
- 5.31
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- Funders
- department of Anaesthesia, Critical Care, and Pain Medicine at the Royal Infirmary of Edinburgh
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis. https://app.cochat.ai/papers/10.1016/j.ijmedinf.2019.103971
- The paper
- Young, I. J. B., Luz, S., & Lone, N. (2019). A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis. International Journal of Medical Informatics. https://doi.org/10.1016/j.ijmedinf.2019.103971
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.