Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood
The DOI matches its Crossref record, with a small difference in title or year; 115 of the 117 references it cites were found in a registry or an index.
Abstract
Background Visual and auditory signs of patient functioning have long been used for clinical diagnosis, treatment selection, and prognosis. Direct measurement and quantification of these signals can aim to improve the consistency, sensitivity, and scalability of clinical assessment. Currently, we investigate if machine learning-based computer vision (CV), semantic, and acoustic analysis can capture clinical features from free speech responses to a brief interview 1 month post-trauma that accurately classify major depressive disorder (MDD) and posttraumatic stress disorder (PTSD). Methods N = 81 patients admitted to an emergency department (ED) of a Level-1 Trauma Unit following a life-threatening traumatic event participated in an open-ended qualitative interview with a para-professional about their experience 1 month following admission. A deep neural network was utilized to extract facial features of emotion and their intensity, movement parameters, speech prosody, and natural language content. These features were utilized as inputs to classify PTSD and MDD cross-sectionally. Results Both video- and audio-based markers contributed to good discriminatory classification accuracy. The algorithm discriminates PTSD status at 1 month after ED admission with an AUC of 0.90 (weighted average precision = 0.83, recall = 0.84, and f1-score = 0.83) as well as depression status at 1 month after ED admission with an AUC of 0.86 (weighted average precision = 0.83, recall = 0.82, and f1-sc…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“From the forest plot highlights studies with notably higher AUC scores, particularly Reece et al, 52 Kirinde Gamaarachchige and Inkpen, 84 and Schultebraucks et al. 43”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [43]
In the citing paper
“The latter study incorporates a broader set of linguistic and acoustic features, including pitch frame counts, potentially enhancing predictive accuracy.”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [43]
In the citing paper
“SHAP was implemented in 4 studies spanning dementia detection [9], auditory verbal hallucination assessment [52], psychiatric disorder classification [51], and TBI [53].”
From Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review (2026) · cites this paper as [51]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 1×
“It is an additive feature attribution method that uses kernel functions and currently the gold standard to interpret deep neural networks (Lundberg & Lee, 2017).”
MethodBreiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325 · cited 1×
“Data were preprocessed using R package caret (Breiman, 1996; Kuhn, 2008; Kuhn & Johnson, 2013).”
MethodGauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
(2016) · Cited by 11,679
The expression of the emotions in man and animals.
Darwin, C. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 11,657
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881 · cited 1×
“We used k -fold cross-validation with 10 folds in the training set to decrease the risk of overfitting (Stone, 1974).”
MethodBuilding Predictive Models in R Using the caret Package
Kuhn, M. (2008) · Journal of Statistical Software · doi:10.18637/jss.v028.i05 · Cited by 8,863 · cited 1×
“Data were preprocessed using R package caret (Breiman, 1996; Kuhn, 2008; Kuhn & Johnson, 2013).”
MethodInsel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., Sanislow, C., & Wang, P. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.09091379 · Cited by 7,067 · cited 1×
“Our results also suggest implications for the fields ’ ability to put a clinical focus on discrete behavioral and physiological dimensions as metrics of risk and treatment response consistent with the research domain criteria approach (Cuthbert & Insel, 2013; Insel, 2014; Insel et al., 2010).”
Result
Research works printed but not linked: 6
- 10.1037/0021-843x.109.1.26 · doi:10.1037/0021-843x.109.1.26 · Paper page
- 2013, Diagnostic and statistical manual of mental disorders (DSM-5®)
- Eaton, 2004, The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults, 363
- Pennebaker, 2015, The development and psychometric properties of LIWC2015
- Ekman, 1978, Facial action coding system: Investigator's guide
- Carmi, 2020, Management of terrorism induced stress – guideline for the golden hours, 175
Cited by
Malgaroli, M., Hull, T. D., Zech, J. M., & Althoff, T. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02592-2 · Cited by 201
“Five deep learning studies mitigated this limitation by utilizing an interpretability algorithm to elucidate their models [71, 79, 85, 104, 122].”
MethodKhoo, L. S., McNaney, R., Lim, M. K., & Chong, C. Y. (2024) · Sensors · doi:10.3390/s24020348 · Cited by 242
Resilience to potential trauma and adversity through regulatory flexibility
Bonanno, G. A., Chen, S., & Galatzer‐Levy, I. R. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00233-5 · Cited by 194
“Similarly, training using reappraisal has led to consistent reductions in negative affect over several days 169,170 .”
BackgroundSelf-citationThe Psychedelic Future of Post-Traumatic Stress Disorder Treatment
Zaretsky, T. G., Jagodnik, K. M., Barsic, R., Antonio, J. H., Bonanno, P. A., MacLeod, C., Pierce, C., Carney, H., Morrison, M. T., Saylor, C. F., et al. (2024) · Current Neuropharmacology · doi:10.2174/1570159x22666231027111147 · Cited by 104
Zarate, D., Stavropoulos, V., Ball, M., Collier, G. D. S., & Jacobson, N. C. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04013-y · Cited by 96
“For example, Schultebraucks et al. [162] observed a predictive accuracy of AUC = 0.86, with lin - guistic features, voice prosody, facial features of emotion, and movement features (such as pupil dilation) as the most important predictors.”
BackgroundInfluentialJiang, Z., Seyedi, S. S., Griner, E. L., Abbasi, A., Rad, A. B., Kwon, H., Cotes, R. O., & Clifford, G. D. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3352075 · Cited by 103
An insight into diagnosis of depression using machine learning techniques: a systematic review
Bhadra, S., & Kumar, C. J. (2022) · Current Medical Research and Opinion · doi:10.1080/03007995.2022.2038487 · Cited by 90
Heyat, M. B. B., Akhtar, F., Munir, F., Sultana, A., Muaad, A. Y. M., Gul, I., Sawan, M., Asghar, W., Iqbal, S. M. A., Baig, A. A., et al. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01346-x · Cited by 62
“Furthermore, in [40], mainly focused on visual and auditory markers of patients functioning for clinical diagnosis, therapy selection, and prognosis, detection method.”
Othmani, A., Brahem, B., Haddou, Y., & Mustaqeem, M. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3312172 · Cited by 57
“…score for seven discrete emotions, a peak of expressivity, facial expressivity index, and expressivity peak count for PTSD classification [59], while to analyze the movement, head movement, attentiveness, and pupil dilation rate are extracted from raw images in the video using OpenFace…”
BackgroundMethodInfluentialIdentification of Diagnostic Markers for Major Depressive Disorder Using Machine Learning Methods
Shu, Z., Bao, Z., Zhao, X., Xu, M., Li, M. D., & Yang, Z. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.645998 · Cited by 52
Fusing features of speech for depression classification based on higher-order spectral analysis
Miao, X., Li, Y., Wen, M., Liu, Y., Julian, I. N., & Guo, H. (2022) · Speech Communication · doi:10.1016/j.specom.2022.07.006 · Cited by 49
Chen, X., & Pan, Z. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18126441 · Cited by 46
“So, voice data are an ideal data source for depression screening [29,30].”
BackgroundMachine Learning and the Digital Measurement of Psychological Health
Galatzer‐Levy, I. R., & Onnela, J.-P. “. (2023) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-080921-073212 · Cited by 48
Self-citationWang, J., Ouyang, H., Jiao, R., Cheng, S., Zhang, H., Shang, Z., Jia, Y., Yan, W., Wu, L., & Liu, W. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01117-5 · Cited by 46
“Its excellent processing ability for high-latitude data will play a prominent role in future PTSD research 52 .”
InfluentialDigital Phenotyping for Differential Diagnosis of Major Depressive Episode: Narrative Review
Ettore, É., Müller, P., Hinze, J., Riemenschneider, M. J., Benoît, M., Giordana, B., Postin, D., Hurlemann, R., Lecomte, A., Musiol, M., et al. (2023) · JMIR Mental Health · doi:10.2196/37225 · Cited by 48
“Moreover, in speech content, trauma narrative shows an increased use of first-person singular pronouns [106,107] and changes in narrative coherence [108,109].”
InfluentialCakmak, A. S., Alday, E. A. P., Poian, G. D., Rad, A. B., Metzler, T. J., Neylan, T. C., House, S. L., Beaudoin, F. L., An, X., Stevens, J. S., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3053909 · Cited by 38
“90 for predicting PTSD one month after ED enrollment [14].”
BackgroundMahayossanunt, Y., Nupairoj, N., Hemrungrojn, S., & Vateekul, P. (2023) · Sensors · doi:10.3390/s23239402 · Cited by 42
“Based on facial features, movement intensity, speech, and text with 81 patients’ personal collection data sets, studies about posttraumatic stress disorder (PTSD) and major depressive disorder (MDD) have been conducted [22].”
BackgroundIntelligent Internet of Medical Things for Depression: Current Advancements, Challenges, and Trends
Heyat, M. B. B., Adhikari, D., Akhtar, F., Parveen, S., Zeeshan, H. M., Ullah, H., Chen, Y.-H., Wang, L., & Sawan, M. (2025) · International Journal of Intelligent Systems · doi:10.1155/int/6801530 · Cited by 42
Applications of Speech Analysis in Psychiatry
Dikaios, K., Rempel, S., Dumpala, S. H., Oore, S., Kiefte, M., & Uher, R. (2023) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000356 · Cited by 35
Ćosić, K., Popović, S., & Wiederhold, B. K. (2024) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2023.0737 · Cited by 33
Schultebraucks, K., Yadav, V., & Galatzer‐Levy, I. R. (2020) · Digital Biomarkers · doi:10.1159/000512394 · Cited by 36
“For more information on the extraction of the video, audio, content, and movement features, please see Schultebraucks et al. [17].”
BackgroundSelf-citationSystematic review of machine learning in PTSD studies for automated diagnosis evaluation
Wu, Y., Mao, K., Dennett, L., Zhang, Y., & Chen, J. (2023) · npj Mental Health Research · doi:10.1038/s44184-023-00035-w · Cited by 30
Schultebraucks, K., & Chang, B. (2020) · Experimental Neurology · doi:10.1016/j.expneurol.2020.113526 · Cited by 28
“The potential of digital phenotyping based on deep learning has been explored by using face, voice and speech content to classify PTSD and depression (Schultebraucks et al., 2020e).”
BackgroundSelf-citationEnhanced large language models for effective screening of depression and anxiety
Liu, J. M., Gao, M., Sabour, S., Chen, Z., Huang, M., & Lee, T. M. C. (2025) · Communications Medicine · doi:10.1038/s43856-025-01158-1 · Cited by 35
“The classi fi cation performance was evaluated using weighted F1 45 , which indicates the harmonic mean of precision and recall depending on each class ’ s sample size.”
AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients
Ćosić, K., Popović, S., Šarlija, M., Kesedžić, I., Gambiraža, M., Dropuljić, B., Mijić, I., Henigsberg, N., & Jovanović, T. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.782866 · Cited by 29
Sentiment Analysis on Social MediaPreprint
Yadav, J. (2023) · Qeios · doi:10.32388/yf9x04 · Cited by 30
Čihařová, M., Amarti, K., van Breda, W., Peng, X., Lorente-Català, R., Funk, B., Hoogendoorn, M., Koutsouleris, N., Fusar‐Poli, P., Karyotaki, E., et al. (2024) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.06.002 · Cited by 25
“Laboratory experiments struggle to recruit samples of a suf fi cient size compared with studies in other populations, probably due to design requirements, and often recruit actors rather than general population participants or patients.”
BackgroundMalgaroli, M., & Schultebraucks, K. (2020) · European Psychologist · doi:10.1027/1016-9040/a000423 · Cited by 25
Self-citationDigital Measurement of Mental Health: Challenges, Promises, and Future Directions
Abbas, A., Schultebraucks, K., & Galatzer‐Levy, I. R. (2021) · Psychiatric Annals · doi:10.3928/00485713-20201207-01 · Cited by 23
Self-citationAn overview of diagnostics and therapeutics using large language models
Malgaroli, M., & McDuff, D. (2024) · Journal of Traumatic Stress · doi:10.1002/jts.23082 · Cited by 22
Sawadogo, M. A. L., Pala, F., Singh, G., Selmi, I., Puteaux, P., & Othmani, A. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17203-x · Cited by 20
“In [8], audio-visual features comprising of face, voice, speech content and movement were extracted from medical interview recordings and fed into a neural network for PTSD and MDD (Major Depressive Disorder) prediction.”
Method Matters: Enhancing Voice‐Based Depression Detection With a New Data Collection Framework
Vilenchik, D., Cwikel, J., Ezra, Y., Hausdorff, T., Lazarov, M., Sergienko, R., Abramovitz, R., Schmidt, I., & Perez, A. S. (2025) · Depression and Anxiety · doi:10.1155/da/4839334 · Cited by 19
Everaert, J., Benisty, H., Polack, R. G., Joormann, J., & Mishne, G. (2022) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000775 · Cited by 20
Parselmouth for bioacoustics: automated acoustic analysis in Python
Jadoul, Y., de Boer, B., & Ravignani, A. (2023) · Bioacoustics · doi:10.1080/09524622.2023.2259327 · Cited by 18
Lamichhane, B., Moukaddam, N. J., & Sabharwal, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-69739-z · Cited by 18
“New features identification in future studies should also explore alternative sensing modalities, such as non-contact sensing 38,39 in home-based monitoring setup, to understand how a broader sensing modality compares for mental health applications.”
Di, Y., Rahmani, E., Mefford, J., Wang, J., Ravi, V., Gorla, A., Alwan, A. A., Kendler, K. S., Zhu, T., & Flint, J. (2024) · Molecular Psychiatry · doi:10.1038/s41380-024-02877-y · Cited by 20
“Many studies have sought to develop features from pitch as a biomarker for depression [17 – 26], but none have, to date, achieved suf fi cient accuracy and precision for clinical utility.”
Machine learning algorithms for predicting PTSD: a systematic review and meta-analysis
Vali, M., Motahari-Nezhad, H., Kovács, L., & Gandomi, A. H. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02754-2 · Cited by 20
“Most of these articles (65%, n = 15) originated from researchers in the United States [11, 31–33, 35–39, 42, 44–48] and China (17%, n = 4) [27, 28, 30, 40].”
InfluentialGupta, R. K., & Sinha, R. (2023) · IEEE Access · doi:10.1109/access.2023.3297091 · Cited by 16
“In a recent work [3], the authors have performed the detection of MDD and PTSD on audio-video data collected through face-to-face interviews from the population admitted to the emergency department of a trauma center.”
Mehta, A., & Yang, W. (2023) · International Conference on Video and Image Processing (ICVIP) · doi:10.1145/3639390.3639392 · Cited by 20
“Socially assistive robots, medical diagnosis of disorders such as PTSD [50], and software usability testing [23] in particular rely heavily on accurate emotion recognition.”
BackgroundComparison of classifiers using robust features for depression detection on Bahasa Malaysia speech
Hashim, N. N. W. N., Basri, N. A., Ezzi, M. A.-E. A., & Hashim, N. M. H. N. (2022) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v11.i1.pp238-253 · Cited by 15
Ganguly, C., Nayak, S., & Gupta, A. K. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-91196-2.00016-8 · Cited by 17
Malgaroli, M., Hull, T. D., & Schultebraucks, K. (2021) · Psychiatric Annals · doi:10.3928/00485713-20201203-01 · Cited by 12
Self-citationHinduja, S., Darzi, A., Ertuğrul, I. Ö., Provenza, N. R., Gadot, R., Storch, E. A., Sheth, S. A., Goodman, W. K., & Cohn, J. F. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3395117 · Cited by 15
“Machine learning from behavioral features has been used widely to detect depression [3], [75], [76], [77], [78] and to some extent anxiety [54], post-traumatic stress disorder (PTSD) [79], [80], and suicidality [3].”
Novel Deep Learning-Based Vocal Biomarkers for Stress Detection in Koreans
Namkung, J., Kim, S. M., Cho, W. I., Yoo, S. Y., Min, B., Lee, S. Y., Lee, J.-H., Park, H., Back, S., Yun, J., et al. (2024) · Psychiatry Investigation · doi:10.30773/pi.2024.0131 · Cited by 11
An Emotion-Driven Vocal Biomarker-Based PTSD Screening Tool
Quatieri, T. F., Wang, J., Williamson, J. R., DeLaura, R. A., Talkar, T., Solomon, N. P., Kuchinsky, S. E., Eitel, M. M., Brickell, T. A., Lippa, S. M., et al. (2023) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2023.3284798 · Cited by 12
Machine yearning: How advances in computational methods lead to new insights about reactions to loss
Malgaroli, M., Maccallum, F., & Bonanno, G. A. (2021) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2021.05.003 · Cited by 11
Self-citationPellegrin, G., Ricka, N., Fompeyrine, D. A., Rohaly, T., Enders, L. R., & Roy, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-91916-x · Cited by 9
Blekić, W., D’Hondt, F., Shalev, A. Y., & Schultebraucks, K. (2025) · Nature Mental Health · doi:10.1038/s44220-024-00365-4 · Cited by 11
“In addition, 13 studies used resampling techniques 26,30,38–40,43,45–47, 50– 52, PTSD was assessed using the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnostic criteria in 19 studies: 4 used the Clinician Administrated PTSD Scale IV (CAPS-IV) 27,33,45,49 , 6…”
InfluentialSelf-citationA Comprehensive Review of Datasets for Clinical Mental Health AI SystemsPreprint
Mandal, A., Adhikary, P. K., Arnaout, H., Gurevych, I., & Chakraborty, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.09809 · Cited by 15
Wang, L., Liu, R., Wang, Y., Xu, X., Zhang, R., Wei, Y., Zhu, R., Zhang, X., & Wang, F. (2024) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-023-09612-3 · Cited by 8
“…methods using acoustic features have been used in the diagnosis of MDD (Di et al., 2021), and the aim of applying deep learning is not to accurately model such theories but, more modestly, to find stable probabilistic patterns in a data-driven manner (Valiant, 1984; Schultebraucks et al., 2020).”
InfluentialDia, M., Khodabandelou, G., & Othmani, A. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108439 · Cited by 8
Li, C., Zhang, K., Lin, Q., Huang, S. H., Cheng, W., Lei, Y., Zhao, X., & Zhao, J. (2025) · BMC Medicine · doi:10.1186/s12916-025-04101-2 · Cited by 8
Tasouj, P. E. M., Soysal, G., Eroğul, O., & Yetkin, S. (2025) · Diagnostics · doi:10.3390/diagnostics15111414 · Cited by 7
“Another study [46] introduced a DL-based method that analyzes facial expressions, movement parameters, speech prosody, and natural language content for the diagnosis of PTSD and major depressive disorder (MDD).”
Deep learning-based detection of depression by fusing auditory, visual and textual clues
Xu, C., Chen, Y., Tao, Y., Xie, W., Liu, X., Lin, Y., Liang, C., Du, F., Lin, Z., & Shi, C. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119860 · Cited by 11
Post-traumatic stress disorder
Ressler, K. J., Rothbaum, B. O., Schnurr, P. P., Binder, E., Moreland-Capuia, A. R., Nievergelt, C. M., Koenen, K. C., Seedat, S., Shalev, A. Y., Marmar, C. R., et al. (2026) · Nature Reviews Disease Primers · doi:10.1038/s41572-026-00701-1 · Cited by 8
“…which are objective and not subject to these biases, including developing blood tests 150–152 , physiology measures 153,154 and voice marker tests, which use audiometric tools (for example, recording on a smart phone) as biometric indicators of certain aspects of psychiatric disorders 155 .”
Self-citationEnhancing PTSD Outcome Prediction with Ensemble Models in Disaster Contexts
Siddiqua, A., Oni, A. M., Miah, A. S. M., & Shin, J. (2025) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecce64574.2025.11013125 · Cited by 7
“Various combinations of risk factors were evaluated, achieving prediction accuracies between 66% and 80%.”
Speech Recognition-Based Prediction for Mental Health and Depression: A Review
Gaikwad, P. P., & Venkatesan, M. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-99-5180-2_2 · Cited by 7
Čihařová, M., Amarti, K., van Breda, W., Gevonden, M. J., Ghassemi, S., Kleiboer, A. M., Vinkers, C. H., Sep, M. S. C., Trofimova, S., Cooper, A. C., et al. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1548287 · Cited by 10
Predicting affective engagement and mental strain from prosodic speech features
Yache, V. P., Moradbakhti, L., Neuner, I., & Veselinović, T. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1656292 · Cited by 6
“…successfully used audio-based markers from free speech responses one month post-trauma to accurately classify major depressive disorder (MDD) and post-traumatic stress disorder (PTSD), demonstrating the potential of vocal biomarkers for early mental health diagnosis following traumatic events (6).”
InfluentialSun, P., Liu, H., Zhao, Y., Hao, N., Deng, Z., & Zhao, W. (2025) · Environment International · doi:10.1016/j.envint.2025.109427 · Cited by 6
Stress Diagnosis Using Deep Learning Techniques
Gautam, R., & Sharma, M. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-2535-1_50 · Cited by 6
Rachakonda, L., & Bipin, K. C. (2022) · IEEE International Symposium on Smart Electronic Systems (iSES) · doi:10.1109/ises54909.2022.00151 · Cited by 8
“Another deep learning-based approach has been proposed to classify the mental disorder as depression or PTSD by performing audio and video analyses are proposed [18].”
MethodMachine Unlearning for Speaker-Agnostic Detection of Gender-Based Violence Condition in Speech
Reyner-Fuentes, E., Rituerto-González, E., & Peláez-Moreno, C. (2025) · Applied Sciences · doi:10.3390/app152212270 · Cited by 4
Sun, P., Liu, H., Zhao, Y., Hao, N., Deng, Z., & Zhao, W. (2024) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2024.134651 · Cited by 4
Hinduja, S., Darzi, A., Ertuğrul, I. Ö., Provenza, N. R., Gadot, R., Storch, E., Sheth, S. A., Goodman, W. K., & Cohn, J. F. (2024) · doi:10.36227/techrxiv.23256119.v2 · Cited by 4
Quillivic, R., Auxéméry, Y., Gayraud, F., Dayan, J., & Mesmoudi, S. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf075 · Cited by 3
Identifying PTSD sex-based patterns through explainable artificial intelligence in biometric data
García-Valdez, A. A., Román-Godínez, I., Salido-Ruiz, R. A., & Torres-Ramos, S. (2024) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-024-00485-y · Cited by 3
“However, something that Marmar et al. (2019) and other authors such as Schultebraucks et al. (2022); Gupta et al. (2022) did not consider is the possibility of finding differences when analyzing the speech of men and women separately, which is reflected in our results.”
InfluentialMohamed, I. I., El-Wakad, M. T., Shafie, K. A., Aboamer, M. A., & Mohamed, N. A. R. (2024) · Applied Computer Science · doi:10.35784/acs-2024-41 · Cited by 3
Menne, F., Schäfer, S., Linz, N., Tröger, J., Schwed, L., Ettore, É., & König, A. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116486 · Cited by 3
Day-to-day dynamics of facial emotion expressions in posttraumatic stress disorder
Ringwald, W. R., Feltman, S. M., Schwartz, H. A., Samaras, D., Khudari, C., Luft, B. J., & Kotov, R. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.03.109 · Cited by 6
“On the other hand, a clear picture hasn ’ t emerged on the specific emotional expressions related to PTSD, with conflicting findings on which emotional expressions are the strongest markers (Fujiwara et al., 2015; Scherer et al., 2014; Schultebraucks et al., 2022).”
Major depressive disorder: early detection using deep learning and pupil diameter
Mohamed, I. I., El-Wakad, M. T., Shafie, K. A., Aboamer, M. A., & Mohamed, N. A. R. (2024) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v35.i2.pp916-932 · Cited by 3
“Schultebraucks et al. [17] employed a deep neural network to extract facial features, speech prosody, movement parameters (e.g., pupil dilation), and natural language content to classify individuals with MDD.”
BackgroundTHE BEHAVIOR ANALYSIS OF DEEP LEARNING MODEL IN THE TREATMENT OF DEPRESSION RESEARCH
YANG, J., & Wang, S. (2025) · Journal of Mechanics in Medicine and Biology · doi:10.1142/s0219519425400494 · Cited by 3
Precision Psychiatry Approach to Posttraumatic Stress Response
Schultebraucks, K., & Shalev, A. Y. (2021) · Psychiatric Annals · doi:10.3928/00485713-20201203-02 · Cited by 3
Self-citationMeaney, T., Yadav, V., Galatzer‐Levy, I. R., & Bryant, R. A. (2025) · JMIR Mental Health · doi:10.2196/83575 · Cited by 3
“The XGBoost classification model had an average overall F 1 -score of 0.78, which is lower than the average F 1 -score achieved by the classification model for PTSD built using a similar approach [29].”
InfluentialSelf-citationMeasurement of differential activation by heart-rate-variability for youth MDD discrimination
Li, C., Yang, Y., Wang, W., Li, H., Mai, Y., & Zhao, J. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.02.006 · Cited by 2
Gupta, R. K., & Sinha, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48309-7_50 · Cited by 2
JTA: Joint networks with tailored attention for speech depression detection
Li, X.-H., Liu, Z., Liu, C.-L., Zhong, B., Chen, J., She, J., & Hirota, K. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114617 · Cited by 1
The Language of Emotions: Algorithmic Reasoning for Stress and Mood Identification
Noronha, V. S., Varuni, C. V., & K, P. K. M. (2025) · International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI) · doi:10.1109/icoici65217.2025.11252594 · Cited by 1
“Schultebraucks et al. [18] studied the classification of post-traumatic stress disorder and depression using deep learning.”
Sahin, B., & Obeid, A. (2025) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet63943.2025.11472155 · Cited by 1
“Similarly, study [42] integrated image, audio, and text data, achieving high accuracy for stress (AUC: 90”
Hilberdink, C. E., Huibregtse, M. E., Hinojosa, C. A., Dean, G. S., van Zuiden, M., & Schultebraucks, K. (2026) · Journal of Traumatic Stress · doi:10.1002/jts.70108 · Cited by 1
Self-citationHilberdink, C. E., Zhao, Y., McKernan, S. M., Gershov, S., Müller, V., Wall, S. P., & Schultebraucks, K. (2026) · NPP—Digital Psychiatry and Neuroscience · doi:10.1038/s44277-026-00067-y · Cited by 1
Self-citationXiaohan, L. I. U., Minglong, C. H. E. N., & Guo, J. (2022) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2022.00851 · Cited by 3
Sex-Based Speech Pattern Recognition for Post-traumatic Stress Disorder
García-Valdez, A. A., Román-Godínez, I., Salido-Ruiz, R. A., & Torres-Ramos, S. (2023) · World Congress on Medical Physics and Biomedical Engineering, September 7 - 12, 2009, Munich, Germany · doi:10.1007/978-3-031-46933-6_21 · Cited by 1
Sentiment Analysis on Social MediaPreprint
Yadav, J. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/d6wv2 · Cited by 1
Day-to-day dynamics of facial emotion expressions in posttraumatic stress disorderPreprint
Ringwald, W. R., Feltman, S. M., Schwartz, H. A., Samaras, D., Khudari, C., Luft, B. J., & Kotov, R. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6fqg9 · Cited by 1
Hinduja, S., Darzi, A., Ertuğrul, I. Ö., Provenza, N. R., Gadot, R., Storch, E., Sheth, S. A., Goodman, W. K., & Cohn, J. F. (2024) · doi:10.36227/techrxiv.23256119.v3 · Cited by 1
Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review
Ebraheem, M., Toghranegar, J., Bridge2AI-Voice Consortium, Bensoussan, Y., & Templeton, J. M. (2026) · Journal of Medical Internet Research · doi:10.2196/83790
“Audio representation in the literature included raw audio [49], low-level acoustic descriptors (eg, jitter, shimmer, and harmonic-to-noise ratio) [51,60], raw time-frequency representations (eg, spectrograms and mel-spectrograms) [6,48,50,82], coefficient-based features (eg, MFCCs) [7,11, 56,82],…”
InfluentialPrevalence of depression and suicidal ideation in trauma patients in western Rajasthan
Kumar, S., Verma, K., & Baniya, G. C. (2022) · Panacea Journal of Medical Sciences · doi:10.18231/j.pjms.2022.055
Application of Deep Learning in Vocal Music Teaching
Zheng, L., & Cui, L. (2023) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns.2023.1.00457
Incremental Validity of Language-Based Assessments of Personality in World Trade Center RespondersPreprint
Oltmanns, J. R., Schwartz, H. A., Ruggero, C. J., Son, Y., Vu, H., Gill, G., Miao, J., Waszczuk, M. A., Clouston, S., Bromet, E. J., et al. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7graf
Schultebraucks, K. (2023) · Trauma und Gewalt · doi:10.21706/tg-17-4-284
Self-citationYu, Z., Crane, C. A., Chen, L., Testa, M., & Zheng, Z. (2024) · ACM Transactions on Computing for Healthcare · doi:10.1145/3664826
“Schultebraucks et al. [53] designed a deep learning-based framework to catch facial features of emotion and related intensities to facilitate the detection of major depressive disorder and posttraumatic stress disorder.”
A Computer Vision Perspective on Emotional Intelligence
Vertan, C., Florea, L., Florea, C., & Racoviţeanu, A. (2024) · Intelligent systems reference library · doi:10.1007/978-3-031-63337-9_7
Patrick, K. S., Burke, E. M., Gunstad, J., & Spitznagel, M. B. (2024) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2024.2432655
“…digital, automatic measurement of daily, naturalistic facial expressions of emotion in the context of major depressive disorder (Abbas et al., 2021; Schultebraucks et al., 2022) and schizophrenia (Abbas et al., 2022; Galatzer-Levy et al., 2020) but has not yet been employed in the context of…”
Review on Detecting Facial Emotions Using Deep Learning for Mental Health Applications
Veligeti, M. S. R. K., & Pavithra, L. K. (2024) · International Journal of Image and Graphics · doi:10.1142/s0219467826500439
Pandey, A. K., Jain, T., Fatma, E., & Raj, H. (2025) · doi:10.71058/jodac.v9i1006
Incremental Validity of Language-Based Assessments of Personality in World Trade Center RespondersPreprint
Oltmanns, J. R., Schwartz, H. A., Ruggero, C. J., Son, Y., Vu, H., Gill, G., Miao, J., Waszczuk, M. A., Clouston, S., Bromet, E. J., et al. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7graf_v1
Psychological Crisis Feature Extraction Based on Deep Learning Algorithm
Chen, H., & Li, H. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-7999-7_22
Predicting Traumatic Brain Injury Post-Trauma Using Temporal Attention on Sleep-Wake Data
Saghafi, S., Li, Q., Neylan, T. C., Thomas, T. T., Stevens, J. S., Jovanović, T., Germine, L. T., Bucher, M. A., Huibregtse, M. E., Linnstaedt, S. D., et al. (2025) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2025.3592009 · Cited by 1
“Advances inmachinelearninganddeeplearninghavesignificantlycon-tributedtoourunderstandinganddiagnosisofTBI,leveraging diversedatasetssuchasneuroimagingdata[38],[39],[40],[41] and audio-based detection techniques [42], [43], [44].”
Zhao, J. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251364421
Metadata
- Authors
- Schultebraucks, K., Yadav, V., Shalev, A. Y., Bonanno, G. A., & Galatzer-Levy, I. R.
- Affiliations
- Columbia University Irving Medical Center (US)
- AiCure (United States) (US)
- New York University (US)
- Columbia University (US)
- Corresponding author
- Schultebraucks, Katharina
- Venue
- Psychological Medicine
- Publisher
- Cambridge University Press (CUP)
- Published
- 2020-08-03
- Volume
- 52
- Issue
- 5
- Pages
- 957-967
- ISSN
- 0033-2917 (ISSN-L)
- 1469-8978
- OpenAlex
- W3047024470
- Semantic Scholar
- fccf0932af2f598ff567f7f61fed92aa7a7217d1
- Cited by
- 108
- References
- 117
- Citations by year
- 2026: 10 · 2025: 31 · 2024: 24 · 2023: 20 · 2022: 12 · 2021: 8 · 2020: 3
- FWCI
- 10.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (K01MH102415)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Social Sciences › Psychology › Clinical Psychology › Posttraumatic Stress Disorder Research
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- post-traumatic stress disorder, major depressive disorder, trauma (physical injury), facial emotion recognition, prosody, acoustic analysis, computer vision, natural language processing, digital biomarkers, emergency department patients
- MeSH terms
- Deep Learning, Arousal, Depression, Depressive Disorder, Major, Major Depressive Disorder, Humans, Stress Disorders, Post-Traumatic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.