Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.

Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach

Open access

Published version of a preprint

Abstract

Background: Suicide is an important public health concern in the United States and around the world. There has been significant work examining machine learning approaches to identify and predict intentional self-harm and suicide using existing data sets. With recent advances in computing, deep learning applications in health care are gaining momentum. Objective: This study aimed to leverage the information in clinical notes using deep neural networks (DNNs) to (1) improve the identification of patients treated for intentional self-harm and (2) predict future self-harm events. Methods: We extracted clinical text notes from electronic health records (EHRs) of 835 patients with International Classification of Diseases (ICD) codes for intentional self-harm and 1670 matched controls who never had any intentional self-harm ICD codes. The data were divided into training and holdout test sets. We tested a number of algorithms on clinical notes associated with the intentional self-harm codes using the training set, including several traditional bag-of-words–based models and 2 DNN models: a convolutional neural network (CNN) and a long short-term memory model. We also evaluated the predictive performance of the DNNs on a subset of patients who had clinical notes 1 to 6 months before the first intentional self-harm event. Finally, we evaluated the impact of a pretrained model using Word2vec (W2V) on performance. Results: The area under the receiver operating characteristic curve (AUC) for the CNN on the phenotyping task, that is, the detection of intentional self-harm in clinical notes concurrent with the events was 0.999, with an F1 score of 0.985. In the predictive task, the CNN achieved the highest performance with an AUC of 0.882 and an F1 score of 0.769. Although pretraining with W2V shortened the DNN training time, it did not improve performance. Conclusions: The strong performance on the first task, namely, phenotyping based on clinical notes, suggests that such models could be used effectively for surveillance of intentional self-harm in clinical text in an EHR. The modest performance on the predictive task notwithstanding, the results using DNN models on clinical text alone are competitive with other reports in the literature using risk factors from structured EHR data.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/17784

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 A visualization of a sample of relevant words derived from the Word2vec model reduced into two dimensions using t-distributed stochastic neighbor embedding. V1=variable 1; V2=variable 2.
Figure 1 A visualization of a sample of relevant words derived from the Word2vec model reduced into two dimensions using t-distributed stochastic neighbor embedding. V1=variable 1; V2=variable 2.CC BY · PubMed Central
Figure 2 The area under the receiver operating characteristic curve for training on the 2012 to 2017 data set and testing on the holdout test set (2018-2019) using (1) International Classification of Diseases labels and (2) gold standard labels. AUC: area under the receiver operating characteristic curve; ICD: International Classification of Diseases; CNNr: convolutional neural network with randomly initialized word embeddings; CNNw: convolutional neural network with Word2vec word embeddings; LSTMr: long short-term memory with randomly initialized word embedding; LSTMw: long short-term memory with Word2vec word embedding.
Figure 2 The area under the receiver operating characteristic curve for training on the 2012 to 2017 data set and testing on the holdout test set (2018-2019) using (1) International Classification of Diseases labels and (2) gold standard labels. AUC: area under the receiver operating characteristic curve; ICD: International Classification of Diseases; CNNr: convolutional neural network with randomly initialized word embeddings; CNNw: convolutional neural network with Word2vec word embeddings; LSTMr: long short-term memory with randomly initialized word embedding; LSTMw: long short-term memory with Word2vec word embedding.CC BY · PubMed Central
Figure 3 The mean area under the receiver operating characteristic curve and 95% CI for models trained on notes preceding the first intentional self-harm visits in patients presenting during the 2012 to 2017 time frame and tested on notes preceding the first intentional self-harm visits in patients presenting during the 2018 to 2019 time frame. The differences in performance were all significant (P<.001) except for the difference between the LSTMr and LSTMw. AUC: area under the receiver operating characteristic curve; CNNr: convolutional neural network with randomly initialized word embeddings; CNNw: convolutional neural network with Word2vec word embeddings; LSTMr: long short-term memory with randomly initialized word embedding; LSTMw: long short-term memory with Word2vec word embedding.
Figure 3 The mean area under the receiver operating characteristic curve and 95% CI for models trained on notes preceding the first intentional self-harm visits in patients presenting during the 2012 to 2017 time frame and tested on notes preceding the first intentional self-harm visits in patients presenting during the 2018 to 2019 time frame. The differences in performance were all significant (P<.001) except for the difference between the LSTMr and LSTMw. AUC: area under the receiver operating characteristic curve; CNNr: convolutional neural network with randomly initialized word embeddings; CNNw: convolutional neural network with Word2vec word embeddings; LSTMr: long short-term memory with randomly initialized word embedding; LSTMw: long short-term memory with Word2vec word embedding.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 50 references

Research works printed but not linked: 14

  1. Kochanek, KD, 2017, NCHS Data Brief, 1
  2. 2016, Surveillance Report 2016 – Self-Harm in Over 8s: Short-Term Management and Prevention of Recurrence (2004) NICE Guideline CG16 and Self-Harm in Over 8s: Long Term Management (2011) NICE Guideline CG133
  3. Haerian, K, 2012, AMIA Annu Symp Proc, 2012, 1244
  4. Downs, J, 2017, AMIA Annu Symp Proc, 2017, 641
  5. MikolovTChenKCorradoGDeanJEfficient estimation of word representations in vector spacearXiv201301162018-11-20https://arxiv.org/pdf/1301.3781.pdf
  6. DevlinJChangMLeeKToutanovaKBERT: pre-training of deep bidirectional transformers for language understandingarXiv201905242020-06-10https://www.aclweb.org/anthology/N19-1423.pdf
  7. Banerjee, I, 2017, AMIA Annu Symp Proc, 2017, 411
  8. Hedegaard, H, 2018, Natl Health Stat Report, 1
  9. McCallumANigamKA comparison of event models for Naive Bayes text classificationAAAI-98 Workshop 'Learning for Text Categorization'19982020-06-10https://www.cs.cmu.edu/~knigam/papers/multinomial-aaaiws98.pdf
  10. van der Maaten, L, 2008, J Mach Learn Res, 9, 2579
  11. LeeJDernoncourtFSequential short-text classification with recurrent and convolutional neural networksarXiv201603112020-06-10https://www.aclweb.org/anthology/N16-1062.pdf
  12. KingmaDBaJAdam: A method for stochastic optimizationarXiv20192020-06-10https://arxiv.org/pdf/1412.6980.pdf
  13. VuNTAdelHGuptaPSchützeHCombining recurrent and convolutional neural networks for relation classificationAssociation for Computational Linguistics201605242020-06-10https://www.aclweb.org/anthology/N16-1065.pdf
  14. MullenbachJWiegreffeSDukeJSunJEisensteinJExplainable prediction of medical codes from clinical textarXiv201804162020-06-10https://arxiv.org/pdf/1802.05695.pdf

Cited by

See all 44

Metadata

Authors
Obeid, J. S., Dahne, J., Christensen, S., Howard, S., Crawford, T., Frey, L. J., Stecker, T., & Bunnell, B. E.
Affiliations
  • Medical University of South Carolina (US)
  • University of South Florida (US)
Corresponding author
Obeid, Jihad S
Venue
JMIR Medical Informatics
Published
2020-07-30
Volume
8
Issue
7
Pages
e17784
ISSN
2291-9694 (ISSN-L)
OpenAlex
W3046293795
Cited by
44
References
50
Citations by year
2026: 3 · 2025: 7 · 2024: 10 · 2023: 11 · 2022: 7 · 2021: 5 · 2020: 1
FWCI
2.36
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Medical Informatics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • U.S. Department of Veterans Affairs (I21 HX002700)
  • National Institutes of Health (UL1TR001450)
  • National Institute of Mental Health (MH118482, K23MH118482)
  • National Institute on Drug Abuse (K23DA045766, DA045766)
  • National Institute of Diabetes and Digestive and Kidney Diseases (P30DK123704)
  • National Center for Advancing Translational Sciences (TR001450)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
Keywords
intentional self-harm, suicide risk prediction, clinical notes, electronic health records, deep neural network, convolutional neural networks, long short-term memory, Word2Vec

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach. https://app.cochat.ai/papers/10.2196/17784
The paper
Obeid, J. S., Dahne, J., Christensen, S., Howard, S., Crawford, T., Frey, L. J., Stecker, T., & Bunnell, B. E. (2020). Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach. JMIR Medical Informatics. https://doi.org/10.2196/17784

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.