The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach
Open accessPublished version of a preprint
Abstract
Background: Suicide is an important public health concern in the United States and around the world. There has been significant work examining machine learning approaches to identify and predict intentional self-harm and suicide using existing data sets. With recent advances in computing, deep learning applications in health care are gaining momentum. Objective: This study aimed to leverage the information in clinical notes using deep neural networks (DNNs) to (1) improve the identification of patients treated for intentional self-harm and (2) predict future self-harm events. Methods: We extracted clinical text notes from electronic health records (EHRs) of 835 patients with International Classification of Diseases (ICD) codes for intentional self-harm and 1670 matched controls who never had any intentional self-harm ICD codes. The data were divided into training and holdout test sets. We tested a number of algorithms on clinical notes associated with the intentional self-harm codes using the training set, including several traditional bag-of-words–based models and 2 DNN models: a convolutional neural network (CNN) and a long short-term memory model. We also evaluated the predictive performance of the DNNs on a subset of patients who had clinical notes 1 to 6 months before the first intentional self-harm event. Finally, we evaluated the impact of a pretrained model using Word2vec (W2V) on performance. Results: The area under the receiver operating characteristic curve (AUC) for the CNN on the phenotyping task, that is, the detection of intentional self-harm in clinical notes concurrent with the events was 0.999, with an F1 score of 0.985. In the predictive task, the CNN achieved the highest performance with an AUC of 0.882 and an F1 score of 0.769. Although pretraining with W2V shortened the DNN training time, it did not improve performance. Conclusions: The strong performance on the first task, namely, phenotyping based on clinical notes, suggests that such models could be used effectively for surveillance of intentional self-harm in clinical text in an EHR. The modest performance on the predictive task notwithstanding, the results using DNN models on clinical text alone are competitive with other reports in the literature using risk factors from structured EHR data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“More complex EMR-based screening, bypassing gatekeepers, may improve identification of higher risk patients.(43–47)”
From Improving Suicide Prevention Through Evidence-Based Strategies: A Systematic Review (2021) · cites this paper as [47]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · cited 2×
“The concatenated strings from concurrent notes for this sample were imported into REDCap (Research Electronic Data Capture) [38] and made available for review and labeling by the reviewers on our research team, which included 3 clinical psychologists, a psychiatry resident, a medical student, and a pediatrician.”
Method(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 3×
“Pretraining models using these methods provide syntactic and semantic word similarities expressed in a multidimensional vector space with the potential for improving classifications based on neural networks and reducing computational cost [28].”
BackgroundMethodAn introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 1×
“The performance metrics for all experiments, including area under the receiver operating characteristic (ROC) curve (AUC), were calculated in R using the caret [52] and pROC [53] packages.”
MethodIntroduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Classification and regression trees
Praagman, J. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90321-2 · Cited by 10,053
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999 · cited 1×
“The traditional text classification models included naïve Bayes [42]; decision tree classifier [43] with a maximum depth of 20; random forest (RF) [44] with 201 trees and the number of variables randomly sampled as candidates at each split (mtry=150); SVM [45] type 1 with a radial basis kernel [46]; and a simple multi…”
MethodMatching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007) · Political Analysis · doi:10.1093/pan/mpl013 · Cited by 4,205 · cited 1×
“The matching was performed using the nearest neighbor method in the MatchIt package in R [37].”
Method
Research works printed but not linked: 14
- Kochanek, KD, 2017, NCHS Data Brief, 1
- 2016, Surveillance Report 2016 – Self-Harm in Over 8s: Short-Term Management and Prevention of Recurrence (2004) NICE Guideline CG16 and Self-Harm in Over 8s: Long Term Management (2011) NICE Guideline CG133
- Haerian, K, 2012, AMIA Annu Symp Proc, 2012, 1244
- Downs, J, 2017, AMIA Annu Symp Proc, 2017, 641
- MikolovTChenKCorradoGDeanJEfficient estimation of word representations in vector spacearXiv201301162018-11-20https://arxiv.org/pdf/1301.3781.pdf
- DevlinJChangMLeeKToutanovaKBERT: pre-training of deep bidirectional transformers for language understandingarXiv201905242020-06-10https://www.aclweb.org/anthology/N19-1423.pdf
- Banerjee, I, 2017, AMIA Annu Symp Proc, 2017, 411
- Hedegaard, H, 2018, Natl Health Stat Report, 1
- McCallumANigamKA comparison of event models for Naive Bayes text classificationAAAI-98 Workshop 'Learning for Text Categorization'19982020-06-10https://www.cs.cmu.edu/~knigam/papers/multinomial-aaaiws98.pdf
- van der Maaten, L, 2008, J Mach Learn Res, 9, 2579
- LeeJDernoncourtFSequential short-text classification with recurrent and convolutional neural networksarXiv201603112020-06-10https://www.aclweb.org/anthology/N16-1062.pdf
- KingmaDBaJAdam: A method for stochastic optimizationarXiv20192020-06-10https://arxiv.org/pdf/1412.6980.pdf
- VuNTAdelHGuptaPSchützeHCombining recurrent and convolutional neural networks for relation classificationAssociation for Computational Linguistics201605242020-06-10https://www.aclweb.org/anthology/N16-1065.pdf
- MullenbachJWiegreffeSDukeJSunJEisensteinJExplainable prediction of medical codes from clinical textarXiv201804162020-06-10https://arxiv.org/pdf/1802.05695.pdf
Cited by
Improving Suicide Prevention Through Evidence-Based Strategies: A Systematic Review
Mann, J. J., Michel, C. A., & Auerbach, R. P. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20060864 · Cited by 655
Rathnayaka, P., Mills, N., Burnett, D., Silva, D. D., Alahakoon, D., & Gray, R. J. (2022) · Sensors · doi:10.3390/s22103653 · Cited by 202
“This detection is performed by the NLU engine in the chatbot during the conversation, by computing an alert score based on the frequency and intensity of words, phrases, and emotions representative of critical danger and self-harm [58,59].”
MethodMachine learning and the prediction of suicide in psychiatric populations: a systematic review
Pigoni, A., Delvecchio, G., Turtulici, N., Madonna, D., Pietrini, P., Cecchetti, L., & Brambilla, P. (2024) · Translational Psychiatry · doi:10.1038/s41398-024-02852-9 · Cited by 175
Sim, J.-A., Huang, X., Horan, M. R., Stewart, C. M., Robison, L. L., Hudson, M. M., Baker, J. N., & Huang, I. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102701 · Cited by 105
Use of unstructured text in prognostic clinical prediction models: a systematic review
Seinen, T. M., Fridgeirsson, E. A., Ioannou, S., Jeannetot, D. Y. B., John, L. H., Kors, J. A., Markus, A. F., Pera, V., Rekkas, A., Williams, R. D., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac058 · Cited by 102
Metadata
- Authors
- Obeid, J. S., Dahne, J., Christensen, S., Howard, S., Crawford, T., Frey, L. J., Stecker, T., & Bunnell, B. E.
- Affiliations
- Medical University of South Carolina (US)
- University of South Florida (US)
- Corresponding author
- Obeid, Jihad S
- ORCID
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2020-07-30
- Volume
- 8
- Issue
- 7
- Pages
- e17784
- ISSN
- 2291-9694 (ISSN-L)
- PMC
- PMC7426805
- OpenAlex
- W3046293795
- Cited by
- 44
- References
- 50
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 10 · 2023: 11 · 2022: 7 · 2021: 5 · 2020: 1
- FWCI
- 2.36
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- U.S. Department of Veterans Affairs (I21 HX002700)
- National Institutes of Health (UL1TR001450)
- National Institute of Mental Health (MH118482, K23MH118482)
- National Institute on Drug Abuse (K23DA045766, DA045766)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK123704)
- National Center for Advancing Translational Sciences (TR001450)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Keywords
- intentional self-harm, suicide risk prediction, clinical notes, electronic health records, deep neural network, convolutional neural networks, long short-term memory, Word2Vec
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach. https://app.cochat.ai/papers/10.2196/17784
- The paper
- Obeid, J. S., Dahne, J., Christensen, S., Howard, S., Crawford, T., Frey, L. J., Stecker, T., & Bunnell, B. E. (2020). Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach. JMIR Medical Informatics. https://doi.org/10.2196/17784
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.