A natural language processing and deep learning approach to identify child abuse from pediatric electronic medical records
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Child physical abuse is a leading cause of traumatic injury and death in children. In 2017, child abuse was responsible for 1688 fatalities in the United States, of 3.5 million children referred to Child Protection Services and 674,000 substantiated victims. While large referral hospitals maintain teams trained in Child Abuse Pediatrics, smaller community hospitals often do not have such dedicated resources to evaluate patients for potential abuse. Moreover, identification of abuse has a low margin of error, as false positive identifications lead to unwarranted separations, while false negatives allow dangerous situations to continue. This context makes the consistent detection of and response to abuse difficult, particularly given subtle signs in young, non-verbal patients. Here, we describe the development of artificial intelligence algorithms that use unstructured free-text in the electronic medical record—including notes from physicians, nurses, and social workers—to identify children who are suspected victims of physical abuse. Importantly, only the notes from time of first encounter (e.g.: birth, routine visit, sickness) to the last record before child protection team involvement were used. This allowed us to develop an algorithm using only information available prior to referral to the specialized child protection team. The study was performed in a multi-center referral pediatric hospital on patients screened for abuse within five different locations between 2015 and 2019. Of 1123 patients, 867 records were available after data cleaning and processing, and 55% were abuse-positive as determined by a multi-disciplinary team of clinical professionals. These electronic medical records were encoded with three natural language processing (NLP) algorithms—Bag of Words (BOW), Word Embeddings (WE), and Rules-Based (RB)—and used to train multiple neural network architectures. The BOW and WE encodings utilize the full free-text, while RB selects crucial phrases as identified by physicians. The best architecture was selected by average classification accuracy for the best performing model from each train-test split of a cross-validation experiment. Natural language processing coupled with neural networks detected cases of likely child abuse using only information available to clinicians prior to child protection team referral with average accuracy of 0.90±0.02 and average area under the receiver operator characteristic curve (ROC-AUC) 0.93±0.02 for the best performing Bag of Words models. The best performing rules-based models achieved average accuracy of 0.77±0.04 and average ROC-AUC 0.81±0.05, while a Word Embeddings strategy was severely limited by lack of representative embeddings. Importantly, the best performing model had a false positive rate of 8%, as compared to rates of 20% or higher in previously reported studies. This artificial intelligence approach can help screen patients for whom an abuse concern exists and streamline the identification of patients who may benefit from referral to a child protection team. Furthermore, this approach could be applied to develop computer-aided-diagnosis platforms for the challenging and often intractable problem of reliably identifying pediatric patients suffering from physical abuse.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312 · cited 7×
“The application of NLP to EMR free-text is particularly promising because the EMR is a complete record, including clinical impressions, social and medical history, summaries of diagnostic tests and studies, and longitudinal documentation of the patient’s course.”
BackgroundInfluentialMachine learning algorithm validation with a limited sample size
Vabalas, A., Gowen, E., Poliakoff, E., & Casson, A. J. (2019) · PLOS ONE · doi:10.1371/journal.pone.0224365 · Cited by 1,901
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710
Desautels, T., Calvert, J., Hoffman, J., Jay, M., Kerem, Y., Shieh, L., Shimabukuro, D., Chettipally, U., Feldman, M. D., Barton, C., et al. (2016) · JMIR Medical Informatics · doi:10.2196/medinform.5909 · Cited by 509
Cited by
Artificial Intelligence -based technologies in nursing: A scoping literature review of the evidence
von Gerich, H., Moen, H., Block, L. J., Chu, C. H., DeForest, H., Hobensack, M., Michalowski, M., Mitchell, J., Nibber, R., Olalia, M. A., et al. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.104153 · Cited by 360
“…et al., 2019 , Lee and Lin, 2020 , Liao et al., 2015 , Oh et al., 2014 , Zamzmi et al., 2019 6.5 Level IV (e.g. descriptive correlational study) Annapragada et al., 2021 , Back et al., 2016 , Chun et al., 2021 , Ginestra et al., 2019 , Hur et al., 2019 , Kang et al., 2018 , Long et al., 2016 ,…”
InfluentialMay Artificial Intelligence Influence Future Pediatric Research?—The Case of ChatGPT
Corsello, A., & Santangelo, A. (2023) · Children · doi:10.3390/children10040757 · Cited by 66
“used NLP to identify child abuse from EMRs [12].”
MethodPerrier, E., Rifai, M., Terzic, A., Dubois, C., & Cohen, J. F. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.1065957 · Cited by 37
“Beyond convolutional neural networks, other AI-enabled diagnostic applications include, for instance, improvement in diagnosing autism (8), identification of child abuse from medical records (9), and natural language processing systems to assist clinicians with detecting rare conditions (10).”
Metadata
- Authors
- Annapragada, A. V., Donaruma-Kwoh, M. M., Annapragada, A. V., & Starosolski, Z. A.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-02-26
- Volume
- 16
- Issue
- 2
- Pages
- e0247404
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W3130411038
- Cited by
- 61
- References
- 4
- FWCI
- 7.98
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.