Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood
The DOI matches its Crossref record, with a small difference in title or year; 115 of the 117 references it cites were found in a registry or an index.
Abstract
Background Visual and auditory signs of patient functioning have long been used for clinical diagnosis, treatment selection, and prognosis. Direct measurement and quantification of these signals can aim to improve the consistency, sensitivity, and scalability of clinical assessment. Currently, we investigate if machine learning-based computer vision (CV), semantic, and acoustic analysis can capture clinical features from free speech responses to a brief interview 1 month post-trauma that accurately classify major depressive disorder (MDD) and posttraumatic stress disorder (PTSD). Methods N = 81 patients admitted to an emergency department (ED) of a Level-1 Trauma Unit following a life-threatening traumatic event participated in an open-ended qualitative interview with a para-professional about their experience 1 month following admission. A deep neural network was utilized to extract facial features of emotion and their intensity, movement parameters, speech prosody, and natural language content. These features were utilized as inputs to classify PTSD and MDD cross-sectionally. Results Both video- and audio-based markers contributed to good discriminatory classification accuracy. The algorithm discriminates PTSD status at 1 month after ED admission with an AUC of 0.90 (weighted average precision = 0.83, recall = 0.84, and f1-score = 0.83) as well as depression status at 1 month after ED admission with an AUC of 0.86 (weighted average precision = 0.83, recall = 0.82, and f1-sc…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“From the forest plot highlights studies with notably higher AUC scores, particularly Reece et al, 52 Kirinde Gamaarachchige and Inkpen, 84 and Schultebraucks et al. 43”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [43]
In the citing paper
“The latter study incorporates a broader set of linguistic and acoustic features, including pitch frame counts, potentially enhancing predictive accuracy.”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [43]
In the citing paper
“SHAP was implemented in 4 studies spanning dementia detection [9], auditory verbal hallucination assessment [52], psychiatric disorder classification [51], and TBI [53].”
From Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review (2026) · cites this paper as [51]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 1×
“It is an additive feature attribution method that uses kernel functions and currently the gold standard to interpret deep neural networks (Lundberg & Lee, 2017).”
MethodBreiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325 · cited 1×
“Data were preprocessed using R package caret (Breiman, 1996; Kuhn, 2008; Kuhn & Johnson, 2013).”
MethodGauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
(2016) · Cited by 11,679
The expression of the emotions in man and animals.
Darwin, C. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 11,657
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881 · cited 1×
“We used k -fold cross-validation with 10 folds in the training set to decrease the risk of overfitting (Stone, 1974).”
MethodBuilding Predictive Models in R Using the caret Package
Kuhn, M. (2008) · Journal of Statistical Software · doi:10.18637/jss.v028.i05 · Cited by 8,863 · cited 1×
“Data were preprocessed using R package caret (Breiman, 1996; Kuhn, 2008; Kuhn & Johnson, 2013).”
MethodInsel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., Sanislow, C., & Wang, P. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.09091379 · Cited by 7,067 · cited 1×
“Our results also suggest implications for the fields ’ ability to put a clinical focus on discrete behavioral and physiological dimensions as metrics of risk and treatment response consistent with the research domain criteria approach (Cuthbert & Insel, 2013; Insel, 2014; Insel et al., 2010).”
Result
Research works printed but not linked: 6
- 10.1037/0021-843x.109.1.26 · doi:10.1037/0021-843x.109.1.26 · Paper page
- 2013, Diagnostic and statistical manual of mental disorders (DSM-5®)
- Eaton, 2004, The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults, 363
- Pennebaker, 2015, The development and psychometric properties of LIWC2015
- Ekman, 1978, Facial action coding system: Investigator's guide
- Carmi, 2020, Management of terrorism induced stress – guideline for the golden hours, 175
Cited by
Malgaroli, M., Hull, T. D., Zech, J. M., & Althoff, T. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02592-2 · Cited by 201
“Five deep learning studies mitigated this limitation by utilizing an interpretability algorithm to elucidate their models [71, 79, 85, 104, 122].”
MethodKhoo, L. S., McNaney, R., Lim, M. K., & Chong, C. Y. (2024) · Sensors · doi:10.3390/s24020348 · Cited by 242
Resilience to potential trauma and adversity through regulatory flexibility
Bonanno, G. A., Chen, S., & Galatzer‐Levy, I. R. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00233-5 · Cited by 196
“Similarly, training using reappraisal has led to consistent reductions in negative affect over several days 169,170 .”
BackgroundSelf-citationThe Psychedelic Future of Post-Traumatic Stress Disorder Treatment
Zaretsky, T. G., Jagodnik, K. M., Barsic, R., Antonio, J. H., Bonanno, P. A., MacLeod, C., Pierce, C., Carney, H., Morrison, M. T., Saylor, C. F., et al. (2024) · Current Neuropharmacology · doi:10.2174/1570159x22666231027111147 · Cited by 104
Zarate, D., Stavropoulos, V., Ball, M., Collier, G. D. S., & Jacobson, N. C. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04013-y · Cited by 96
“For example, Schultebraucks et al. [162] observed a predictive accuracy of AUC = 0.86, with lin - guistic features, voice prosody, facial features of emotion, and movement features (such as pupil dilation) as the most important predictors.”
BackgroundInfluential
Metadata
- Authors
- Schultebraucks, K., Yadav, V., Shalev, A. Y., Bonanno, G. A., & Galatzer-Levy, I. R.
- Affiliations
- Columbia University Irving Medical Center (US)
- AiCure (United States) (US)
- New York University (US)
- Columbia University (US)
- Corresponding author
- Schultebraucks, Katharina
- Venue
- Psychological Medicine
- Publisher
- Cambridge University Press (CUP)
- Published
- 2020-08-03
- Volume
- 52
- Issue
- 5
- Pages
- 957-967
- ISSN
- 0033-2917 (ISSN-L)
- 1469-8978
- OpenAlex
- W3047024470
- Semantic Scholar
- fccf0932af2f598ff567f7f61fed92aa7a7217d1
- Cited by
- 108
- References
- 117
- Citations by year
- 2026: 10 · 2025: 31 · 2024: 24 · 2023: 20 · 2022: 12 · 2021: 8 · 2020: 3
- FWCI
- 10.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (K01MH102415)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Social Sciences › Psychology › Clinical Psychology › Posttraumatic Stress Disorder Research
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- post-traumatic stress disorder, major depressive disorder, trauma (physical injury), facial emotion recognition, prosody, acoustic analysis, computer vision, natural language processing, digital biomarkers, emergency department patients
- MeSH terms
- Deep Learning, Arousal, Depression, Depressive Disorder, Major, Major Depressive Disorder, Humans, Stress Disorders, Post-Traumatic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.