CoChat

A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis

Open access
Young, Ian James Bruce, Luz, Saturnino, Lone, Nazir · 2019 · International Journal of Medical Informatics
Cited by 139

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

CONTEXT Adverse events in healthcare are often collated in incident reports which contain unstructured free text. Learning from these events may improve patient safety. Natural language processing (NLP) uses computational techniques to interrogate free text, reducing the human workload associated with its analysis. There is growing interest in applying NLP to patient safety, but the evidence in the field has not been summarised and evaluated to date. OBJECTIVE To perform a systematic literature review and narrative synthesis to describe and evaluate NLP methods for classification of incident reports and adverse events in healthcare. METHODS Data sources included Medline, Embase, The Cochrane Library, CINAHL, MIDIRS, ISI Web of Science, SciELO, Google Scholar, PROSPERO, hand searching of key articles, and OpenGrey. Data items were manually abstracted to a standardised extraction form. RESULTS From 428 articles screened for eligibility, 35 met the inclusion criteria of using NLP to perform a classification task on incident reports, or with the aim of detecting adverse events. The majority of studies used free text from incident reporting systems or electronic health records. Models were typically designed to classify by type of incident, type of medication error, or harm severity. A broad range of NLP techniques are demonstrated to perform these classification tasks with favourable performance outcomes. There are methodological challenges in how these results can be interpreted in a broader context. CONCLUSION NLP can generate meaningful information from unstructured data in the specific domain of the classification of incident reports and adverse events. Understanding what or why incidents are occurring is important in adverse event analysis. If NLP enables these insights to be drawn from larger datasets it may improve the learning from adverse events in healthcare.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.ijmedinf.2019.103971

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 56

  1. Wood, 2005, Mandatory state-based error-reporting systems: current and future prospects, Am. J. Med. Qual., 20, 297, 10.1177/1062860605281850 · DOI
  2. Liang, 2017, Automated classification of multi-labeled patient safety reports: a shift from quantity to quality measure, Stud. Health Technol. Inform., 245, 1070
  3. Govindan, 2010, Automated detection of harm in healthcare with information technology: a systematic review, Qual. Saf. Health Care, 19, e11
  4. Carrell, 2017, Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings, J. Am. Med. Inform. Assoc., 24, 986, 10.1093/jamia/ocx039 · DOI · Paper page
  5. Jacobsson, 2017, Extraction of adverse event severity information from clinical narratives using natural language processing, Pharmacoepidemiol. Drug Saf., 26, 37
  6. Tian, 2017, Automated extraction of vte events from narrative radiology reports in electronic health records: a validation study, Med. Care, 55, e73, 10.1097/MLR.0000000000000346 · DOI
  7. Galvez, 2017, The use of natural language processing on pediatric diagnostic radiology reports in the electronic health record to identify deep venous thrombosis in children, J. Thromb. Thrombolysis, 44, 281, 10.1007/s11239-017-1532-y · doi:10.1007/s11239-017-1532-y
  8. Yim, 2017, Classifying tumor event attributes in radiology reports, J. Assoc. Inf. Sci. Technol., 68, 2662, 10.1002/asi.23937 · DOI
  9. Moore, 2017, Using natural language processing to extract abnormal results from cancer screening reports, J. Patient Saf., 13, 138, 10.1097/PTS.0000000000000127 · DOI
  10. Yetisgen-Yildiz, 2011, Automatic identification of critical follow-up recommendation sentences in radiology reports, AMIA Annu. Symp. Proc. 2011, 1593
  11. Duke, 2010, ADESSA: a real-time decision support service for delivery of semantically coded adverse drug event data, AMIA Annu. Symp. Proc. 2010, 177
  12. Li, 2015, Automated detection of medication administration errors in neonatal intensive care, J. Biomed. Inform., 57, 124, 10.1016/j.jbi.2015.07.012 · DOI
  13. Ni, 2018, Designing and evaluating an automated system for real-time medication administration error detection in a neonatal intensive care unit, J. Am. Med. Inform. Assoc., 25, 555, 10.1093/jamia/ocx156 · DOI
  14. Cai, 2016, Natural language processing to rapidly identify potential signals for adverse events using electronic medical record data: example of arthralgias and vedolizumab, Arthritis Rheumatol., 68, 2802
  15. Nunes, 2016, Assessing occurrence of hypoglycemia and its severity from electronic health records of patients with type 2 diabetes mellitus, Diabetes Res. Clin. Pract., 121, 192, 10.1016/j.diabres.2016.09.012 · DOI
  16. Toyabe, 2015, Characteristics of Inpatient Falls not Reported in an Incident Reporting System, Glob. J. Health Sci., 8, 17, 10.5539/gjhs.v8n3p17 · DOI
  17. Tanushi, 2014, Detection of healthcare-associated urinary tract infection in Swedish electronic health records, Stud. Health Technol. Inform., 207, 330
  18. Jensen, 2017, Analysis of free text in electronic health records for identification of cancer patient trajectories, Sci. Rep., 7, 46226, 10.1038/srep46226 · DOI · Paper page
  19. F. A, 2015, An evaluation of patient safety event report categories using unsupervised topic modeling, Methods Inf. Med., 54, 338, 10.3414/ME15-01-0010 · DOI
  20. Ong, 2010, Automated categorisation of clinical incident reports using statistical text classification, Qual. Saf. Health Care, 19, e55
  21. Gupta, 2015, Automated classification of clinical incident types, Stud. Health Technol. Inform., 214, 87
  22. Melton, 2005, Automated detection of adverse events using natural language processing of discharge summaries, J. Am. Med. Inform. Assoc., 12, 448, 10.1197/jamia.M1794 · DOI · Paper page
  23. Penz, 2007, Automated identification of adverse events related to central venous catheters, J. Biomed. Inform., 40, 174, 10.1016/j.jbi.2006.06.003 · DOI
  24. Ong, 2012, Automated identification of extreme-risk events in clinical incident reports, J. Am. Med. Inform. Assoc., 19, e110, 10.1136/amiajnl-2011-000562 · DOI · Paper page
  25. Murff, 2011, Automated identification of postoperative complications within an electronic medical record using natural language processing, JAMA, 306, 848
  26. Boyce, 2017, Automated screening of emergency department notes for drug-associated bleeding adverse events occurring in older adults, Appl. Clin. Inform., 8, 1022, 10.4338/ACI-2017-02-RA-0036 · doi:10.4338/aci-2017-02-ra-0036
  27. G. J, 2013, Automated validation of patient safety clinical incident classification: macro analysis, Stud. Health Technol. Inform., 188, 52
  28. Fujita, 2013, Detecting effective classes of medical incident reports based on linguistic analysis for common reporting system in Japan, Stud. Health Technol. Inform., 192, 137
  29. Toyabe, 2012, Detecting inpatient falls by using natural language processing of electronic medical records, BMC Health Serv. Res., 12, 448, 10.1186/1472-6963-12-448 · DOI
  30. Han, 2017, Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system, J. Am. Med. Inform. Assoc., 24, 913, 10.1093/jamia/ocx022 · DOI
  31. Benin, 2016, Electronic approaches to making sense of the text in the adverse event reporting system, J. Healthc. Risk Manag., 36, 10, 10.1002/jhrm.21237 · DOI
  32. Liang, 2015, Enhancing patient safety event reporting by K-nearest neighbor classifier, Stud. Health Technol. Inform., 218, 40603
  33. Fong, 2015, Exploring methods for identifying related patient safety events using structured and unstructured data, J. Biomed. Inform., 58, 89, 10.1016/j.jbi.2015.09.011 · DOI
  34. Speroff, 2013, Exploring the frontier of electronic health record surveillance the case of postoperative complications, Med. Care, 51, 509, 10.1097/MLR.0b013e31828d1210 · DOI · Paper page
  35. Gaebel, 2015, Extraction of adverse events from clinical documents to support decision making using semantic preprocessing, Stud. Health Technol. Inform., 216, 1030
  36. Iqbal, 2015, Identification of adverse drug events from free text electronic patient records and information in a large mental health case register, PLoS One, 10, e0134208, 10.1371/journal.pone.0134208 · DOI
  37. Fong, 2017, Integrating natural language processing expertise with patient safety event review committees to improve the analysis of medication events, Int. J. Med. Inform., 104, 120, 10.1016/j.ijmedinf.2017.05.005 · DOI
  38. Weller, 2017, Leveraging electronic health records for predictive modeling of post-surgical complications, Stat. Methods Med. Res., 1
  39. Fujita, 2012, Linguistic analysis of large-scale medical incident reports for patient safety, Stud. Health Technol. Inform., 180, 250
  40. Ravindranath, 2017, Machine learning in automated classification of adverse events in clinical studies of Alzheimer’s disease, Alzheimer’s Dement., 13, P1256, 10.1016/j.jalz.2017.06.1871 · doi:10.1016/j.jalz.2017.06.1871
  41. Liang, 2017, Predicting Harm Scores from Patient Safety Event Reports, Stud. Health Technol. Inform., 245, 1075
  42. W.M M, 2017, Screening electronic health record-related patient safety reports using machine learning, J. Patient Saf., 13, 31, 10.1097/PTS.0000000000000104 · DOI · Paper page
  43. McKnight, 2012, Semi-supervised classification of patient safety event reports, J. Patient Saf., 8, 60, 10.1097/PTS.0b013e31824ab987 · DOI
  44. Wong, 2016, Statistical classification of drug incidents due to look-alike sound-alike mix-ups, Health Informatics J., 22, 276, 10.1177/1460458214555040 · doi:10.1177/1460458214555040
  45. Wong, 2013, Statistical text classifier to detect specific type of medical incidents, Stud. Health Technol. Inform., 192, 1053
  46. Gerdes, 2013, Text mining electronic health records to identify hospital adverse events, Stud. Health Technol. Inform., 192, 1145
  47. Botsis, 2011, Text mining for the vaccine adverse event reporting system: medical text classification using informative feature selection, J. Am. Med. Inform. Assoc., 18, 631, 10.1136/amiajnl-2010-000022 · DOI
  48. Fong, 2017, Using active learning to identify health information technology related patient safety events, Appl. Clin. Inform., 8, 35
  49. Wang, 2017, Using multiclass classification to automate the identification of patient safety incident reports by type and severity, BMC Med. Inform. Decis. Mak., 17, 84, 10.1186/s12911-017-0483-8 · DOI · Paper page
  50. Botsis, 2012, Vaccine adverse event text mining system for extracting features from vaccine safety reports, J. Am. Med. Inform. Assoc., 19, 1011, 10.1136/amiajnl-2012-000881 · DOI
  51. Penz, 2007, Automated identification of adverse events related to central venous catheters, J. Biomed. Inform., 40, 174, 10.1016/j.jbi.2006.06.003 · DOI
  52. Stehman, 1997, Selecting and interpreting measures of thematic classification accuracy, Remote Sens. Environ., 62, 77, 10.1016/S0034-4257(97)00083-7 · DOI · Paper page
  53. Chubak, 2012, Tradeoffs between accuracy measures for electronic health care data algorithms, J. Clin. Epidemiol., 65, 343, 10.1016/j.jclinepi.2011.09.002 · DOI · Paper page
  54. Murdoch, 2013, The Inevitable Application of Big Data to Health Care, JAMA Evid., 309, 1351, 10.1001/jama.2013.393 · DOI · Paper page
  55. Adam, 2017, Data science, learning, and applications to biomedical and health sciences, Ann. N. Y. Acad. Sci., 1387, 5, 10.1111/nyas.13309 · DOI
  56. Young, 2018, Getting the measure of diabetes: the evolution of the national diabetes audit, Pract. Diabetes, 35, 1

Cited by

See all 139

Metadata

Authors
Young, I. J. B., Luz, S., & Lone, N.
Venue
International Journal of Medical Informatics
Publisher
Elsevier BV
Published
2019-12
Volume
132
Pages
103971
ISSN
1386-5056 (ISSN-L)
OpenAlex
W2978315017
Cited by
139
References
2
FWCI
5.31
Type
journal article
Language
en
Open access
green
Licence
  • www.elsevier.com/tdm/userlicense/1.0/ (tdm)
  • www.elsevier.com/legal/tdmrep-license (tdm)
Funders
department of Anaesthesia, Critical Care, and Pain Medicine at the Royal Infirmary of Edinburgh

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis. https://app.cochat.ai/papers/10.1016/j.ijmedinf.2019.103971
The paper
Young, I. J. B., Luz, S., & Lone, N. (2019). A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis. International Journal of Medical Informatics. https://doi.org/10.1016/j.ijmedinf.2019.103971

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.