Artificial intelligence language predictors of two-year trauma-related outcomes
Open accessPublished version of a preprint
Abstract
Background Recent research on artificial intelligence has demonstrated that natural language can be used to provide valid indicators of psychopathology. The present study examined artificial intelligence-based language predictors (ALPs) of seven trauma-related mental and physical health outcomes in responders to the World Trade Center disaster.Methods The responders (N = 174, M age = 55.4 years) provided daily voicemail updates over 14 days. Algorithms developed using machine learning in large social media discovery samples were applied to the voicemail transcriptions to derive ALP scores for several risk factors (depressivity, anxiousness, anger proneness, stress, and personality). Responders also completed self-report assessments of these risk factors at baseline and trauma-related mental and physical health outcomes at two-year follow-up (including symptoms of depression, posttraumatic stress disorder, sleep disturbance, respiratory problems, and GERD).Results Voicemail ALPs were significantly associated with a majority of the trauma-related outcomes at two-year follow-up, over and above corresponding baseline self-reports. ALPs showed significant convergence with corresponding self-report scales, but also considerable uniqueness from each other and from self-report scales.Limitations The study has a relatively short follow-up period relative to trauma occurrence and a limited sample size.Conclusions This study shows evidence that ALPs may provide a novel, objective, and clinically useful approach to forecasting, and may in the future help to identify individuals at risk for negative health outcomes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Words indicating depressive and anxious language patterns were identified as language markers of PTSD symptoms and severity.
“Words indicating depressive and anxious language patterns (related to the DLATK 35 models or LIWC categories) were also identified as language markers of PTSD symptoms and severity.44,60,61”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369 · see citing works
Diagnostic and Statistical Manual of Mental Disorders
Williams, J. B. W., & First, M. B. (2013) · Oxford University Press eBooks · doi:10.1093/acrefore/9780199975839.013.104 · Cited by 109,841 · see citing works
A Global Measure of Perceived Stress
Cohen, S., Kamarck, T., & Mermelstein, R. (1983) · Journal of Health and Social Behavior · doi:10.2307/2136404 · Cited by 32,807 · see citing works
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337 · see citing works
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,649 · see citing works
Personality and the prediction of consequential outcomes.
Ozer, D., & Benet-Martínez, V. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.57.102904.190127 · Cited by 2,224 · see citing works
Soto, C. J., & John, O. P. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000096 · Cited by 2,075 · see citing works
Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700 · see citing works
Self-citationMorin, C. M., & Jarrin, D. C. (2022) · Sleep Medicine Clinics · doi:10.1016/j.jsmc.2022.03.003 · Cited by 907 · see citing works
Automatic personality assessment through social media language.
Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000020 · Cited by 802 · see citing works
Self-citationNeuroticism and common mental disorders: meaning and utility of a complex relationship.
Ormel, J., Jeronimus, B. F., Kotov, R., Riese, H., Bos, E. H., Hankin, B., Rosmalen, J. G. M., & Oldehinkel, A. J. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.04.003 · Cited by 703 · see citing works
Self-citationFacebook language predicts depression in medical records
Eichstaedt, J. C., Smith, R. J., Merchant, R. M., Ungar, L. H., Crutchley, P., Preoţiuc-Pietro, D., Asch, D. A., & Schwartz, H. A. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1802331115 · Cited by 643 · see citing works
Self-citationComorbidity, Impairment, and Suicidality in Subthreshold PTSD
Marshall, R. D., Olfson, M., Hellman, F., Blanco, C., Guardino, M., & Struening, E. L. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.9.1467 · Cited by 578 · see citing works
Diagnostic and Statistical Manual of Mental Disorders
REIFF, M. I., & Feldman, H. M. (2014) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0000000000000017 · Cited by 577 · see citing works
Methods in predictive techniques for mental health status on social media: a critical review
Chancellor, S., & De Choudhury, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0233-7 · Cited by 557 · see citing works
Watson, D., O’Hara, M. W., Naragon-Gainey, K., Koffel, E., Chmielewski, M., Kotov, R., Stasik, S. M., & Ruggero, C. J. (2012) · Assessment · doi:10.1177/1073191112449857 · Cited by 470 · see citing works
Self-citationJakupcak, M., Conybeare, D., Phelps, L. F., Hunt, S. C., Holmes, H. A., Felker, B. L., Klevens, M., & McFall, M. (2007) · Journal of Traumatic Stress · doi:10.1002/jts.20258 · Cited by 444 · see citing works
Initial validation of a diagnostic questionnaire for gastroesophageal reflux disease
Shaw, M. J., Talley, N. J., Beebe, T. J., Rockwood, T. H., Carlsson, R., Adlis, S. A., Fendrick, A. M., Jones, R. H., Dent, J., & Bytzer, P. M. (2001) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2001.03451.x · Cited by 403 · see citing works
Pretrauma risk factors for posttraumatic stress disorder: A systematic review of the literature
DiGangi, J. A., Gomez, D., Mendoza, L., Jason, L. A., Keys, C. B., & Koenen, K. C. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.05.002 · Cited by 320 · see citing works
Persistence of multiple illnesses in World Trade Center rescue and recovery workers: a cohort study
Wisnivesky, J. P., Teitelbaum, S. L., Todd, A. C., Boffetta, P., Crane, M., Crowley, L. E., de la Hoz, R. E., Dellenbaugh, C., Harrison, D. J., Herbert, R., et al. (2011) · The Lancet · doi:10.1016/s0140-6736(11)61180-x · Cited by 300 · see citing works
Self-citationTowards Assessing Changes in Degree of Depression through Facebook
Schwartz, H. A., Eichstaedt, J., Kern, M. L., Park, G., Sap, M., Stillwell, D., Kosinski, M., & Ungar, L. (2014) · Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality · doi:10.3115/v1/w14-3214 · Cited by 256 · see citing works
Self-citationGaining insights from social media language: Methodologies and challenges.
Kern, M. L., Park, G., Eichstaedt, J. C., Schwartz, H. A., Sap, M., Smith, L. K., & Ungar, L. H. (2016) · Psychological Methods · doi:10.1037/met0000091 · Cited by 221 · see citing works
Self-citationMorin, C. M., & Jarrin, D. C. (2013) · Sleep Medicine Clinics · doi:10.1016/j.jsmc.2013.05.002 · Cited by 216 · see citing works
Eichstaedt, J. C., Kern, M. L., Yaden, D. B., Schwartz, H. A., Giorgi, S., Park, G., Hagan, C. A., Tobolsky, V. A., Smith, L. K., Buffone, A. E. K., et al. (2021) · Psychological Methods · doi:10.1037/met0000349 · Cited by 193 · see citing works
Self-citationPerceptions of people with personality disorders based on thin slices of behavior
Oltmanns, T. F., Friedman, J. N. W., Fiedler, E. R., & Turkheimer, E. (2004) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00066-7 · Cited by 189 · see citing works
Bromet, E. J., Hobbs, M. J., Clouston, S. A. P., Gonzalez, A., Kotov, R., & Luft, B. J. (2016) · Psychological Medicine · doi:10.1017/s0033291715002184 · Cited by 165 · see citing works
Self-citationFear and Loathing: A Meta-Analytic Review of the Specificity of Anger in PTSD
Olatunji, B. O., Ciesielski, B. G., & Tolin, D. F. (2009) · Behavior Therapy · doi:10.1016/j.beth.2009.01.004 · Cited by 164 · see citing works
Cohort Profile: World Trade Center Health Program General Responder Cohort
Dasaro, C. R., Holden, W. L., Berman, K. D., Crane, M. A., Kaplan, J. R., Lucchini, R. G., Luft, B. J., Moline, J. M., Teitelbaum, S. L., Tirunagari, U. S., et al. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv099 · Cited by 132 · see citing works
Self-citationDLATK: Differential Language Analysis ToolKit
Schwartz, H. A., Giorgi, S., Sap, M., Crutchley, P., Ungar, L., & Eichstaedt, J. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/d17-2010 · Cited by 130 · see citing works
Self-citationThe use of machine learning techniques in trauma-related disorders: a systematic review
Ramos-Lima, L. F., Waikamp, V., Antonelli-Salgado, T., Passos, I. C., & Freitas, L. H. M. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2019.12.001 · Cited by 98 · see citing works
Evaluating the predictability of medical conditions from social media posts
Merchant, R. M., Asch, D. A., Crutchley, P., Ungar, L., Guntuku, S. C., Eichstaedt, J. C., Hill, S., Padrez, K., Smith, R. J., & Schwartz, H. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215476 · Cited by 96 · see citing works
Self-citationAnger, stress, coping, social support and health: Modelling the relationships
Diong, S. M., Bishop, G. D., Enkelmann, H. C., Tong, E. M. W., Why, Y. P., Ang, J. C. H., & Khader, M. (2005) · Psychology and Health · doi:10.1080/0887044040512331333960 · Cited by 95 · see citing works
Development and Validation of the Faceted Inventory of the Five-Factor Model (FI-FFM)
Watson, D., Nus, E., & Wu, K. D. (2019) · Assessment · doi:10.1177/1073191117711022 · Cited by 83 · see citing works
Knaevelsrud, C., Liedl, A., & Maercker, A. R. A. (2010) · Journal of Health Psychology · doi:10.1177/1359105309360073 · Cited by 75 · see citing works
Natale, V., Léger, D., Bayon, V., Erbacci, A., Tonetti, L., Fabbri, M., & Martoni, M. (2015) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000177 · Cited by 75 · see citing works
Early Linguistic Markers of Trauma-Specific Processing Predict Post-trauma Adjustment
Kleim, B., Horn, A. B., Kraehenmann, R., Mehl, M. R., & Ehlers, A. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00645 · Cited by 74 · see citing works
Caska, C. M., & Renshaw, K. D. (2011) · Anxiety Stress & Coping · doi:10.1080/10615806.2011.638053 · Cited by 60 · see citing works
Posttraumatic Stress Disorder and the Risk of Respiratory Problems in World Trade Center Responders
Kotov, R., Bromet, E. J., Schechter, C. B., Broihier, J., Feder, A., Friedman‐Jiménez, G., Gonzalez, A., Guerrera, K. M., Kaplan, J. R., Moline, J. M., et al. (2015) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000179 · Cited by 53 · see citing works
Self-citationLiu, K., Chen, E. Y. H., Chan, C., Lee, D. T. S., Law, Y. W., Conwell, Y., & Yip, P. S. F. (2006) · Psychological Medicine · doi:10.1017/s0033291706009032 · Cited by 49 · see citing works
Zvolensky, M. J., Farris, S. G., Kotov, R., Schechter, C. B., Bromet, E. J., Gonzalez, A., Vujanovic, A. A., Pietrzak, R. H., Crane, M., Kaplan, J. R., et al. (2015) · Preventive Medicine · doi:10.1016/j.ypmed.2015.03.017 · Cited by 42 · see citing works
Self-citationSon, Y., Clouston, S. A. P., Kotov, R., Eichstaedt, J. C., Bromet, E. J., Luft, B. J., & Schwartz, H. A. (2023) · Psychological Medicine · doi:10.1017/s0033291721002294 · Cited by 42 · see citing works
Self-citationTrauma exposure, posttraumatic stress, and preventive health behaviours: a systematic review
Lee, S. Y., & Park, C. L. (2017) · Health Psychology Review · doi:10.1080/17437199.2017.1373030 · Cited by 32 · see citing works
Waszczuk, M. A., Li, K., Ruggero, C. J., Clouston, S., Luft, B. J., & Kotov, R. (2018) · Annals of Behavioral Medicine · doi:10.1093/abm/kax030 · Cited by 31 · see citing works
Self-citationEichstaedt, J. C., Kern, M. L., Yaden, D. B., et al. (2020) · doi:10.31234/osf.io/t52c6 · Cited by 30 · see citing works
Waszczuk, M. A., Ruggero, C. J., Li, K., Luft, B. J., & Kotov, R. (2018) · Behaviour Research and Therapy · doi:10.1016/j.brat.2018.10.018 · Cited by 18 · see citing works
Self-citationSasso, K. E., & Strunk, D. R. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.04.007 · Cited by 15 · see citing works
Pathway from PTSD to respiratory health: Longitudinal evidence from a psychosocial intervention.
Waszczuk, M. A., Li, X., Bromet, E. J., Gonzalez, A., Zvolensky, M. J., Ruggero, C. J., Luft, B. J., & Kotov, R. (2017) · Health Psychology · doi:10.1037/hea0000472 · Cited by 15 · see citing works
Self-citation#Sad: Twitter Content Predicts Changes in Cognitive Vulnerability and Depressive Symptoms
Sasso, M. P., Giovanetti, A. K., Schied, A. L., Burke, H. H., & Haeffel, G. J. (2019) · Cognitive Therapy and Research · doi:10.1007/s10608-019-10001-6 · Cited by 11 · see citing works
Son, Y., Clouston, S., Kotov, R., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06457 · Cited by 7 · see citing works
Self-citationHouston Strong: Linguistic Markers of Resilience after Hurricane Harvey
Marshall, K., Abate, A. C., & Venta, A. C. (2020) · doi:10.37532/jtsdt.2020.9(2).199 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Kjell, O., Kjell, K., & Schwartz, H. A. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115667 · Cited by 96 · see citing works
Self-citationPsychosocial concerns in burn survivors and their families: A narrative review
Beylin, D., Kornhaber, R. A., & Cleary, M. L. (2025) · Injury · doi:10.1016/j.injury.2025.112626 · Cited by 21 · see citing works
Jin, Z., Hu, J., Bi, D., Zhao, K., & Yu, H. (2025) · JMIR Formative Research · doi:10.2196/78401 · Cited by 7 · see citing works
Ilyinskaya, E., Snæbjarnarson, V., Carlsen, H. K., & Oddsson, B. (2024) · Natural hazards and earth system sciences · doi:10.5194/nhess-24-3115-2024 · Cited by 4 · see citing works
Quillivic, R., Auxéméry, Y., Gayraud, F., Dayan, J., & Mesmoudi, S. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf075 · Cited by 3 · see citing works
Incremental Validity of Language-Based Assessments of Personality in World Trade Center RespondersPreprint
Oltmanns, J. R., Schwartz, H. A., Ruggero, C. J., Son, Y., Vu, H., Gill, G., Miao, J., Waszczuk, M. A., Clouston, S., Bromet, E. J., et al. (2023) · doi:10.31234/osf.io/7graf
Self-citation(2024) · Natural hazards and earth system sciences · doi:10.5194/nhess-2023-6-rc1
(2024) · Natural hazards and earth system sciences · doi:10.5194/nhess-2023-6-rc2
Incremental Validity of Language-Based Assessments of Personality in World Trade Center RespondersPreprint
Oltmanns, J. R., Schwartz, H. A., Ruggero, C. J., Son, Y., Vu, H., Gill, G., Miao, J., Waszczuk, M. A., Clouston, S., Bromet, E. J., et al. (2023) · doi:10.31234/osf.io/7graf_v1
Self-citationNatural Language Processing for PTSD Detection: A Systematic Review and Bibliometric Analysis
Seven, E., & Yücel, E. (2025) · Acadlore Transactions on AI and Machine Learning · doi:10.56578/ataiml040404
Tapia, J., Martínez-Pérez, R.-G., Herrera, Y., & García, M. V. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-031-98890-5_38
Measuring resilience using language modeling: A computational approach to observing resilience
Mahwish, S., Boyd, R. L., Varadarajan, V., Kotov, R., Luft, B. J., Schwartz, H. A., & Clouston, S. (2026) · Journal of Traumatic Stress · doi:10.1002/jts.70046
Self-citationKjell, O., Ganesan, A. V., Boyd, R. L., Oltmanns, J. R., Rivero, A., Feltman, S. M., Carr, M. A., Alves, J., Luft, B. J., Kotov, R., et al. (2026) · Clinical Psychological Science · doi:10.1177/21677026261439026
Self-citation
- Authors
- Oltmanns, J. R., Schwartz, H. A., Ruggero, C., Son, Y., Miao, J., Waszczuk, M., Clouston, S. A. P., Bromet, E. J., Luft, B. J., & Kotov, R.
- Affiliations
- Stony Brook University (US)
- University of North Texas (US)
- Rosalind Franklin University of Medicine and Science (US)
- Corresponding author
- Oltmanns, Joshua R.
- ORCID
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2021-11
- Volume
- 143
- Pages
- 239-245
- ISSN
- 0022-3956 (ISSN-L)
- PMC
- PMC8935804
- OpenAlex
- W3160473765
- Cited by
- 13
- References
- 49
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 4 · 2023: 3
- FWCI
- 1.71
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- National Institute on Alcohol Abuse and Alcoholism (R01AA028032-01)
- National Institute for Occupational Safety and Health (U01OH011321)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › Posttraumatic Stress Disorder Research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Keywords
- natural language processing, World Trade Center responders, disaster responders, post-traumatic stress disorder, depression, sleep disorders, respiratory problems, gastroesophageal reflux disease, stress, anger proneness, anxiety, personality, risk prediction, self-report validation
- MeSH terms
- Anxiety, Artificial Intelligence, Disasters, Humans, Language, Middle Aged, Stress Disorders, Post-Traumatic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
