Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review
Open accessAbstract
OBJECTIVES: Diagnosing post-traumatic stress disorder (PTSD) remains a challenge due to symptom variability and comorbidities. Linguistic analysis offers an innovative approach to identify PTSD symptoms and severity. This systematic review aimed at identifying linguistic features associated with PTSD, assessing the quality and limitations of existing studies, summarizing the predictive performance of identified models, and describing the clinical utility of these models. MATERIALS: A comprehensive search was conducted across multiple databases, resulting in the identification of 593 articles. After screening and eligibility assessment, 58 studies were included. METHODS: Data extraction focused on study characteristics, methodology, and performance metrics. We assessed the risk of bias using the PROBAST and conducted both a narrative synthesis and a meta-analysis. RESULTS: Linguistic features such as pronoun use, emotional valence, cognitive processing words, narrative length, discourse disorganization, temporal orientation, specific lexical fields (death, anxiety, sensory-perception details), and disfluencies were commonly investigated. The meta-analysis revealed a pooled area under the curve of 0.81, indicating the high performance of classification models. However, significant publication bias and heterogeneity were noted. Only 8 studies were rated with a low risk of bias, highlighting common issues such as inadequate control groups, unvalidated linguistic tools, unvalidated diagnosis tools, and low rigor in statistical analysis. DISCUSSION AND CONCLUSIONS: Linguistic markers showed potential for enhancing PTSD diagnoses, but the contemporary research was limited by methodological inconsistencies and biases. Future research should focus on standardized tools, symptom-focused studies, and interdisciplinary collaboration to improve the robustness and clinical applicability of findings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
55 of 101 cited sources checked · 29 supported · 1 flagged · accuracy not yet measured
- Findingcitation
Some studies used suboptimal tools such as LIWC for POS tagging instead of more suitable alternatives like the spaCy or Stanza libraries.
“Technology gaps were also a problem, with some studies using suboptimal tools such as LIWC for POS tagging instead of more suitable alternatives like the spaCy 86 or Stanza 89 libraries.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [86] He, 2017, Automated assessment of patients’ self-narratives for posttraumatic stress disorder screening using natural language processing and text mining, Assessment source
The Discussion (page unavailable) proposes adding grammatical properties and parts of speech in a future study, rather than reporting POS tagging in this study. The Preprocessing section describes stop-word removal and stemming, not LIWC; the paper also does not compare LIWC with spaCy or Stanza. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured - cites [89] Qi, 2020 source
- Findingcitation
In Ng et al, use of somatosensory and perceptual words in genocide testimonies predicted PTSD symptoms years later.
“In Ng et al, 67 the use of somatosensory and perceptual words in genocide testimonies predicted PTSD symptoms years later.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [67] Ng, 2015, Narrative characteristics of genocide testimonies predict posttraumatic stress disorder symptoms years later, Psychol Trauma source
- Findingcitation
Death vocabulary was associated with more PTSD symptoms.
“Not all the studies found an association between death vocabulary and PTSD symptoms or severity, but many 40,45,47,49,56,57,60,62,[64][65][66] were consistent in indicating that death vocabulary was associated with more PTSD symptoms.”
Quote in the paperSubstantial claim3 of 3 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
- cites [45] Kleim, 2018, Early linguistic markers of trauma-specific processing predict post-trauma adjustment, Front Psychiatry source
- cites [47] Todorov, 2020, Context- and subgroup-specific language changes in individuals who develop PTSD after trauma, Front Psychol source
The Results section reports higher death-related word use in some PTSD groups than in controls, including both recent- and past-trauma groups compared with one control group (Figure 1F; page not available). That supports an association with PTSD group membership, but the paper does not establish that greater death vocabulary corresponds to more PTSD symptoms or greater symptom severity. (LLM, conf=0.84)
Experimental — accuracy not yet measured
- Findingcitation
In Reece et al, LabMT was the first predictor in the random forest model to detect PTSD on Twitter data.
“In Reece et al, 52 the LabMT was the first predictor in the random forest model to detect PTSD on Twitter data.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [52] Reece, 2017, Forecasting the onset and course of mental illness with Twitter data, Sci Rep source
- Findingcitation
Kernot et al linked more frequent use of past tense to increased symptom severity.
“However, Kernot et al 76 provided a contrasting perspective, linking a more frequent use of past tense to increased symptom severity.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [76] Kernot, 2021 source
The Results report a higher ‘focuspast’ word score among women with high versus low self-reported PTSD scores (8.54 versus 7.28), and the Discussion describes greater use of words focused on the past among women reporting more mental health symptoms. The paper does not specifically report past-tense usage, so that narrower wording is not established by its finding. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Findingcitation
Emotional valences measured with tools such as LIWC, VADER, LabMT, or NRC emotion lexicon were often associated with PTSD symptoms or severity.
“Emotional valences, measured with tools such as the LIWC, VADER 51 LabMT, or NRC emotion lexicon were often associated with PTSD symptoms or severity.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [51] Hutto, 2014, VADER: a parsimonious rule-based model for sentiment analysis of social media text, ICWSM source
- Findingcitation
Some studies suggested that longer narratives are associated with more severe PTSD symptoms.
“Some studies 40,44,62,64 suggested that longer narratives are associated with more severe PTSD symptoms”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
The Results (Language features statistical analysis; page not provided) associate longer narratives with full or partial PTSD and with hypervigilance and functional-significance symptoms. They do not establish that narrative length increases with the severity of PTSD symptoms, as distinct from their presence or diagnostic status. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [44] Son, 2021, World Trade Center responders in their own words: predicting PTSD symptom trajectories with AI-based language analyses of interviews, Psychol Med source
- cites [62] Alvarez-Conrad, 2001, Linguistic predictors of trauma pathology and physical health, Appl Cogn Psychol source
- Findingcitation
A decrease in first-person plural pronouns was noted, highlighting a possible sense of isolation or disconnection from others.
“On the other side, a decrease in first-person plural pronouns (eg, “we”) was also noted, highlighting a possible sense of isolation or disconnection from others.47,48”
Quote in the paperNot a substantial claim2 of 2 sources checked - Findingcitation
Fewer positive emotions were linked to more PTSD symptoms.
“Conversely, other studies 40,59 found that fewer positive emotions were linked to more PTSD symptoms”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
- cites [59] Manne, 2002, Language use and post-traumatic stress symptomatology in parents of pediatric cancer survivors, J Appl Soc Pyschol source
In ‘Associations Between LIWC Indicators and Post-Traumatic Stress Symptoms,’ Table 2 shows a significant negative correlation between positive-emotion words and avoidance symptoms, consistent with fewer such words being linked to more avoidance. But the correlation with total symptoms is not marked significant, and the accompanying prose says fewer positive-emotion words were associated with less avoidance—the opposite direction. The source therefore does not clearly support the broader claim about PTSD symptoms. (LLM, conf=0.75)
Experimental — accuracy not yet measured
- Findingcitation
VADER was used to detect higher negative sentiment scores in PTSD-related texts.
“VADER was also used to detect higher negative sentiment scores in PTSDrelated texts.53”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [53] Sawalha, 2022, Detecting presence of PTSD using sentiment analysis from text data, Front Psychiatry source
- Findingcitation
A higher number of temporal, spatial, and perception junctures in narratives was associated with higher PTSD symptom severity scores.
“Overall, a higher number of temporal, spatial, and perception junctures in narratives was associated with higher PTSD symptom severity scores.40,66,70,71”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
The Results section on language features associates PTSD with physical-sensation words, longer narratives, and other linguistic markers; the Discussion also mentions perception-related verbs and future tense. Neither reports counts of temporal, spatial, or perception junctures or links those counts to PTSD symptom severity scores (searched for junctures and the three specified types). (LLM, conf=0.72)
Experimental — accuracy not yet measured - cites [66] Auxéméry, 2021, Identification des marques du traumatisme psychique dans le langage parlé: définition de l’échelle diagnostique « SPLIT-10, Ann Méd-Psychol Rev Psychiatr source
- cites [70] Shaw, 2001, Linguistic analysis to assess medically related posttraumatic stress symptoms, Psychosomatics source
- Findingcitation
Increased use of first-person singular pronouns was a common marker associated with PTSD.
“Increased use of first-person singular pronouns (eg, “I”) was a common marker associated with PTSD, suggesting heightened self-focus and immersion in traumatic experiences.”
Quote in the paperSubstantial claim2 of 3 sources checked- cites [44] Son, 2021, World Trade Center responders in their own words: predicting PTSD symptom trajectories with AI-based language analyses of interviews, Psychol Med source
- cites [45] Kleim, 2018, Early linguistic markers of trauma-specific processing predict post-trauma adjustment, Front Psychiatry source
- cites [46] Santoro, 2018, Childbirth narratives of women with posttraumatic stress symptoms in the postpartum period, J Obstet Gynecol Neonatal Nurs source
- Findingcitation
Words indicating depressive and anxious language patterns were identified as language markers of PTSD symptoms and severity.
“Words indicating depressive and anxious language patterns (related to the DLATK 35 models or LIWC categories) were also identified as language markers of PTSD symptoms and severity.44,60,61”
Quote in the paperSubstantial claim2 of 3 sources checked- cites [44] Son, 2021, World Trade Center responders in their own words: predicting PTSD symptom trajectories with AI-based language analyses of interviews, Psychol Med source
- cites [60] Oltmanns, 2021, Artificial intelligence language predictors of two-year trauma-related outcomes, J Psychiatr Res source
- cites [61] Coppersmith, 2014, Measuring post traumatic stress disorder in Twitter, ICWSM. source
- Interpretationcitation
Past tense usage generally suggests psychological distancing from the trauma.
“Past tense usage has generally suggested a psychological distancing from the trauma”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [59] Manne, 2002, Language use and post-traumatic stress symptomatology in parents of pediatric cancer survivors, J Appl Soc Pyschol source
The Discussion says greater past-tense use may indicate that parents had processed and integrated the cancer experience, and reports an association with fewer traumatic-stress symptoms. It does not describe this as psychological distancing; its proposed explanation is confronting the experience rather than avoiding it. The association was significant for parents whose children had completed treatment more than five years earlier, not generally across the sample. (LLM, conf=0.79)
Experimental — accuracy not yet measured - cites [74] Booker, 2018, Narratives in the immediate aftermath of traumatic injury: markers of ongoing depressive and posttraumatic stress disorder symptoms, J Trauma Stress source
- Findingcitation
Most studies suggested that a lower frequency of cognitive processing words correlates with worse PTSD outcomes.
“More precisely, most studies 45,54,62,68,72,73 have suggested that a lower frequency of cognitive processing words correlate with worse PTSD outcomes”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [45] Kleim, 2018, Early linguistic markers of trauma-specific processing predict post-trauma adjustment, Front Psychiatry source
- cites [54] Paquet, 2024, Implications of language use in posttraumatic nightmares on psychological symptoms, Psychol Trauma source
- cites [62] Alvarez-Conrad, 2001, Linguistic predictors of trauma pathology and physical health, Appl Cogn Psychol source
- Findingcitation
Survivors with high levels of PTSD symptoms described more external events in their narratives, including detailed accounts of traumatic settings and sensory experiences.
“Survivors with high levels of PTSD symptoms described more external events in their narratives, which included detailed accounts of the traumatic settings and sensory experiences.69”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [69] Filkuková, 2016, The relationship between posttraumatic stress symptoms and narrative structure among adolescent terrorist-attack survivors, Eur J Psychotraumatology source
The Results (page unavailable) support the finding that adolescent survivors with high posttraumatic stress symptoms described more external events, specifically actions and dialogue. They do not establish that this included more detailed accounts of traumatic settings or sensory experiences; the Results instead report more coded “details” among survivors with low symptom levels. (LLM, conf=0.90)
Experimental — accuracy not yet measured
- Findingcitation
Increased disfluencies in speech are commonly observed in individuals with PTSD.
“Studies 40,80 have indicated that increased disfluencies in speech are commonly observed in individuals with PTSD.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
- cites [80] Gayraud, 2023, Le développement du discours des personnes blessées psychiques: de l’indicible à la métaphore, Ann Méd-Psychol Rev Psychiatr source
- Findingcitation
Other applicability concerns, and so clinical utility, arose from studies using manual annotation for linguistic features, which reduced clinical applicability due to the high cost of manual annotation.
“Other applicability concerns, and so clinical utility, arose from studies using manual annotation for linguistic features, which reduced clinical applicability due to the high cost of manual annotation.50,71,77,80”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [50] Frabetti, 2023, Étude de l’agentivité dans le discours de femmes souffrant de trouble de stress post-traumatique dans les suites de violences conjugales, L’Encéphale source
- cites [71] Gayraud, 2022, Étude des références pronominales à soi dans les témoignages d’événements traumatiques de guerre et d’attentats, Lévolution Psychiatr source
Section 4.1 (p. 10) says the study used manual recoding to classify pronoun uses after software extraction. The Introduction (p. 4) also notes a limitation of automatic pronoun counting, but attributes it to pronoun ambiguity. The paper does not say manual annotation was costly or that its cost reduced clinical applicability. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [77] Hellawell, 2004, A comparison of flashbacks and ordinary autobiographical memories of trauma: content and language, Behav Res Ther source
- Findingcitation
Less coherent narratives, characterized by disjointed and fragmented storytelling, are associated with more severe PTSD symptoms.
“Studies 40,69,79,80 have found that less coherent narratives, characterized by disjointed and fragmented storytelling, are associated with more severe PTSD symptoms.”
Quote in the paperSubstantial claim2 of 3 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
The Results (Language features statistical analysis; page unavailable) associate full or partial PTSD with more repetitive discourse and disfluencies, and associate hypervigilance symptoms with more repetitions and fewer logical connectors. These findings are consistent with aspects of fragmented storytelling, but the paper does not establish that lower narrative coherence is associated with greater overall PTSD symptom severity. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [69] Filkuková, 2016, The relationship between posttraumatic stress symptoms and narrative structure among adolescent terrorist-attack survivors, Eur J Psychotraumatology source
The Abstract (Results) reports that narratives with fewer organized thoughts were associated with higher posttraumatic stress symptoms. But it also says the groups did not differ in narrative fragmentation; the Results likewise found no significant fragmentation difference between high- and low-symptom groups. The source therefore supports the organization aspect, not the claim that fragmented storytelling was associated with more severe symptoms. (LLM, conf=0.87)
Experimental — accuracy not yet measured - cites [79] Jones, 2007, The organisation and content of trauma memories in survivors of road traffic accidents, Behav Res Ther source
- Findingreasoning
Studies 42,61,84,88 reporting only classification performances or using deep learning methods without methods to interpret had very low clinical impact.
“Studies 42,61,84,88 reporting only classification performances or using deep learning methods without methods to interpret had very low clinical impact.”
Quote in the paperSubstantial claim1 of 3 sources checked - Findingcitation
Research has shown an inconsistent relationship between the use of cognitive processing words and PTSD outcome.
“The results of research have shown an inconsistent relationship between the use of cognitive processing words and the outcome of PTSD.”
Quote in the paperSubstantial claim - Findingcitation
Perceptual process words in post-trauma nightmares were related to higher PTSD symptom severity.
“Also, perceptual process words in post-trauma nightmares were related to higher PTSD symptom severity.68”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [68] Paquet, 2020, An examination of the relationship between language use in post-trauma nightmares and psychological sequelae in a treatment seeking population, Int J Dream Res source
- Findingcitation
Increased use of negative emotions in nightmare narratives related to higher PTSD symptom severity.
“An increased use of negative emotions in nightmare narratives also related to higher PTSD symptom severity.54”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [54] Paquet, 2024, Implications of language use in posttraumatic nightmares on psychological symptoms, Psychol Trauma source
- Findingown data
Linguistic features such as pronoun use, emotional valence, cognitive processing words, narrative length, discourse disorganization, temporal orientation, specific lexical fields, and disfluencies were commonly investigated.
“Linguistic features such as pronoun use, emotional valence, cognitive processing words, narrative length, discourse disorganization, temporal orientation, specific lexical fields (death, anxiety, sensory-perception details), and disfluencies were commonly investigated.”
Quote in the paperNot a substantial claim - Findingcitation
Frabetti et al found the association between first-person singular pronoun use and PTSD in the opposite direction, with "I" tending to correlate negatively with HAD-A and HAD-D.
“However, Frabetti et al 50 found the association in the opposite direction (ie, the use of the subject pronoun “I” tended to correlate negatively with the HAD-A and HAD-D).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [50] Frabetti, 2023, Étude de l’agentivité dans le discours de femmes souffrant de trouble de stress post-traumatique dans les suites de violences conjugales, L’Encéphale source
- Findingreasoning
After screening and eligibility assessment, 58 studies were included in the final review.
“Consequently, 58 studies were included in the final review”
Quote in the paperSubstantial claim - Findingcitation
Narratives rich in negative emotion words were linked to worse post-treatment functioning.
“Alvarez-Conrad et al 62 noted that narratives rich in negative emotion words were linked to worse post-treatment functioning.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [62] Alvarez-Conrad, 2001, Linguistic predictors of trauma pathology and physical health, Appl Cogn Psychol source
- Findingcitation
Most studies found that more spatio-sensorial details were associated with PTSD symptoms and severity.
“Most of the studies found that more spatio-sensorial details were associated with PTSD symptoms and severity.”
Quote in the paperSubstantial claim - Findingown data
A total of 14 studies with 19,701 participants were included in the meta-analysis based on the presence of AUC or equivalent measures.
“A total of 14 studies (N_sample ¼ 19 701) were included in our meta-analysis based on the presence or absence of AUC 83 metrics or equivalent measures such as sensitivity and specificity.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [83] Flach
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data
Meta-regression with R = 0.56 examined the impact of sample size and PTSD prevalence within datasets on classification performance.
“Our metaregression (R ¼ 0.56) examined the impact of sample size and PTSD prevalence within datasets on classification performance.”
Quote in the paperSubstantial claim - Findingnone given
Other biases arose from statistical methodological issues, such as failure to report statistical power or failure to perform recalibration of statistical tests.
“Other biases arose from statistical methodological issues, such as failure to report statistical power or failure to perform recalibration of statistical tests.”
Quote in the paperNot a substantial claim - Findingown data
Most biases stemmed from the analysis used, where many studies had a very low event per variable number of linguistic features included in the model relative to the number of PTSD cases in the cohort.
“Finally, most biases stemmed from the analysis used, where many studies had a very low event per variable (EPV) number of linguistic features included in the model relative to the number of PTSD cases in the cohort.”
Quote in the paperSubstantial claim - Findingcitation
Lower readability scores, suggesting simpler language and less complex narrative structures, are associated with more severe, chronic PTSD symptoms.
“Lower readability scores, suggesting simpler language and less complex narrative structures, are associated with more severe, chronic PTSD symptoms.37,38,82”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [37] Amir, 1998, Relationship between trauma narratives and trauma pathology, J Trauma Stress source
- cites [38] Gray, 2001, Complexity of trauma narratives as an index of fragmented memory in PTSD: a critical analysis, Appl Cogn Psychol source
- cites [82] Zoellner, 2002, Peritraumatic dissociative experiences, trauma narratives, and trauma pathology, J Trauma Stress source
- Findingown data
When studies utilizing social media data were excluded, the asymmetry was reduced.
“Notably, when studies utilizing social media data were excluded (right panel), the asymmetry was reduced.”
Quote in the paperNot a substantial claim - Findingown data
The most common weakness concerning the predictor was the use of LIWC without reporting coverage for each category or evaluating LIWC's performance on the specific corpus.
“Concerning the predictor, the most common weakness was the use of LIWC without reporting the coverage for each category or evaluating LIWC's performance on the specific corpus.”
Quote in the paperNot a substantial claim - Findingown data
For most of the studies, the authors had little concern about applicability.
“For most of the studies, we had little concern about applicability (see Table S8).”
Quote in the paperNot a substantial claim - Findingown data
The most common weakness concerning participants was the failure of some studies to satisfactorily incorporate control groups.
“The most common weakness concerning the participants was the failure of some studies to satisfactorily incorporate control groups.”
Quote in the paperNot a substantial claim - Findingown data
In the subgroup excluding social media data (n = 8, N = 8381), the pooled AUC was 0.79 (95% CI 0.75, 0.85).
“The pooled AUC for this subset was slightly lower, at 0.79 (95% CI: 0.75, 0.85).”
Quote in the paperSubstantial claim - Findingown data
The two funnel plots revealed a small asymmetry, supporting the presence of publication bias.
“The 2 funnel plots in Figure 5 illustrate the distribution of studies and revealed a small asymmetry, supporting the presence of publication bias.”
Quote in the paperNot a substantial claim - Findingnone given
Specific linguistic features, such as pronoun use and emotional valence, consistently correlate with PTSD symptoms across different studies.
“Our findings indicate that specific linguistic features, such as pronoun use and emotional valence, consistently correlate with PTSD symptoms across different studies.”
Quote in the paperSubstantial claim - Findingnone given
Death vocabulary, first-person pronouns, and emotional valence were consistent across the literature as linguistic markers associated with PTSD.
“Some markers, such as death vocabulary, firstperson pronouns, and emotional valence, were consistent across the literature”
Quote in the paperSubstantial claim - Findingown data
Funnel plot, forest plot, and statistical tests indicated a publication bias towards positive results.
“but the funnel plot, forest plot, and statistical tests indicated a publication bias towards positive results.”
Quote in the paperSubstantial claim - Findingnone given
Verbal tense and disfluencies were less studied.
“Additionally, verbal tense and disfluencies were less studied.”
Quote in the paperNot a substantial claim - Findingown data
Only 8 studies had a low risk of bias (RoB).
“Our RoB analysis demonstrated that only 8 studies had a low RoB (risk of bias).”
Quote in the paperNot a substantial claim - Findingown data
Pooled AUC varies by study design.
“Subgroup analysis shows that pooled AUC varies by study design”
Quote in the paperNot a substantial claim - Findingnone given
Cognitive processing words and narrative length showed less consensus across the literature.
“while others, like the use of cognitive processing words and narrative length, showed less consensus.”
Quote in the paperSubstantial claim - Findingown data
Low prevalence of PTSD and smaller datasets tend to overestimate classification performance (AUC).
“The results confirm our hypothesis that a low prevalence of PTSD and smaller datasets tend to overestimate classification performance (AUC).”
Quote in the paperSubstantial claim - Findingown data
Meta-analysis revealed an average AUC of 0.81 (95% CI: 0.75, 0.85).
“Our meta-analysis revealed an average AUC of 0.81 (95% CI: 0.75, 0.85)”
Quote in the paperSubstantial claim - Findingnone given
Some studies reported AUCs above 0.9.
“For instance, some studies reported AUCs above 0.9”
Quote in the paperSubstantial claim - Findingown data
Self-reported diagnoses account for 15% of the included studies.
“Notably, self-reported diagnoses account for 15% of the included studies”
Quote in the paperSubstantial claim - Findingown data
Significant heterogeneity was observed, as indicated by the homogeneity test and visual inspection of the Galbraith plot.
“The significant heterogeneity observed, as indicated by the homogeneity test and visual inspection of the Galbraith plot (Figure S6)”
Quote in the paperSubstantial claim - Premisecitation
Diagnosing PTSD remains a challenge due to symptom variability and comorbidities.
“Diagnosing post-traumatic stress disorder (PTSD) remains a challenge due to symptom variability and comorbidities.”
- Also stated in: § Background · plain
“An accurate diagnosis of PTSD is challenging due to its fluctuating symptoms, which can change significantly over time.4”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] Bryant, 2013, A multisite analysis of the fluctuating course of posttraumatic stress disorder, JAMA Psychiatry source
- Also stated in: § Background · plain
- Interpretationreasoning
Spatio-sensorial details in trauma narratives reflect the vivid and intrusive nature of trauma memories.
“These details were interpreted as reflecting the vivid and intrusive nature of trauma memories.”
Quote in the paperSubstantial claim - Interpretationreasoning
Selective reporting bias could contribute, as some studies may only present their most successful models rather than a full range of results.
“Additionally, selective reporting bias could contribute, as some studies may only present their most successful models rather than a full range of results.”
Quote in the paperNot a substantial claim - Interpretationown data
Inclusion of social media data may contribute to a marginal increase in overall classification performance.
“This suggests that the inclusion of social media data may contribute to a marginal increase in the overall classification performances.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The studies with notably higher AUC scores may have inflated the overall pooled AUC.
“These studies may have inflated the overall pooled AUC.”
Quote in the paperSubstantial claim - Interpretationown data
The asymmetry suggests that studies reporting higher AUC values may be overrepresented, potentially due to publication bias.
“This asymmetry suggests that studies reporting higher AUC values may be overrepresented, potentially due to publication bias”
Quote in the paperSubstantial claim - Interpretationreasoning
Contemporary research on linguistic markers for PTSD was limited by methodological inconsistencies and biases.
“but the contemporary research was limited by methodological inconsistencies and biases.”
Quote in the paperSubstantial claim - Interpretationreasoning
Linguistic markers showed potential for enhancing PTSD diagnoses.
“Linguistic markers showed potential for enhancing PTSD diagnoses”
Quote in the paperSubstantial claim- Rests on an assumption: Language provides a promising and readily accessible route to understanding human suffering, making it a valuable tool for identifying PTSD.
- Interpretationcitation
Schultebraucks et al incorporates a broader set of linguistic and acoustic features, including pitch frame counts, potentially enhancing predictive accuracy.
“The latter study incorporates a broader set of linguistic and acoustic features, including pitch frame counts, potentially enhancing predictive accuracy.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [43] Schultebraucks, 2020, Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood, Psychol Med source
- Interpretationreasoning
This variability may affect the generalizability of the findings mentioned above.
“This variability may affect the generalizability of the findings mentioned above.”
Quote in the paperSubstantial claim - Interpretationown data
The pooled AUC of 0.81 suggests high classification performance.
“The pooled AUC of 0.81 suggests high classification performance.”
Quote in the paperSubstantial claim - Interpretationreasoning
The classification performance reported in the meta-analysis may be overestimated.
“the classification performance reported in the metaanalysis should be interpreted with caution, as it may be overestimated.”
Quote in the paperSubstantial claim - Interpretationown data
The significant heterogeneity suggests variability in study-level characteristics, including population demographics, data collection methods, and classification models.
“suggests variability in study-level characteristics, including population demographics, data collection methods, and classification models.”
Quote in the paperSubstantial claim - Interpretationreasoning
The results of this systematic review indicate the potential of linguistic markers for use in the diagnosis of PTSD.
“The results of this systematic review indicate the potential of linguistic markers for use in the diagnosis of PTSD.”
Quote in the paperSubstantial claim - Interpretationreasoning
AUCs above 0.9 are unlikely given the inherent variability of gold-standard metrics.
“which is unlikely given the inherent variability of gold-standard metrics.”
Quote in the paperSubstantial claim - Interpretationreasoning
Over-fitting may limit the reliability of conclusions drawn from these models.
“As a result, over-fitting may limit the reliability of conclusions drawn from these models.”
Quote in the paperSubstantial claim - Interpretationreasoning
Using LIWC categories without confirming their performance in different contexts, such as Twitter (now X), transcriptions, or short writing tasks, can lead to unreliable results.
“using LIWC categories without confirming their performance in different contexts, such as Twitter (now X), transcriptions, or short writing tasks, can lead to unreliable results.”
Quote in the paperSubstantial claim - Interpretationcitation
Use of black box models without interpretative frameworks diminished the practical applicability of findings.
“the use of black box models without interpretative frameworks diminished the practical applicability of findings.”
Quote in the paperSubstantial claim - Interpretationown data
The included data's heterogeneity impacts generalizability.
“Similarly, the included data's heterogeneity impacts generalizability.”
Quote in the paperSubstantial claim - Interpretationreasoning
Lack of standardized tools complicated comparisons across studies, as the methods for measuring speech disorganization varied widely.
“the lack of standardized tools complicated comparisons across studies, as the methods for measuring speech disorganization varied widely, from manual annotations based on clinical grids to readability scores and graph modeling.”
Quote in the paperSubstantial claim - Interpretationnone given
Inadequate study designs that failed to account for comorbidities and identify features that are truly discriminative for PTSD limited clinical relevance.
“Inadequate study designs that failed to account for comorbidities and identify features that are truly discriminative for PTSD limited clinical relevance.”
Quote in the paperSubstantial claim - Interpretationreasoning
Some linguistic features identified as associated with PTSD symptoms may be less specific or statistically significant than expected, particularly for less studied features such as verbal tense and disfluencies.
“This also suggests that some linguistic features identified as associated with PTSD symptoms may be less specific or statistically significant than expected, particularly for less studied features such as verbal tense and disfluencies.”
Quote in the paperSubstantial claim - Interpretationreasoning
Use of unvalidated linguistic tools can lead to spurious results if not adequately controlled.
“Another critical limitation was the use of unvalidated linguistic tools, which can lead to spurious results if not adequately controlled.”
Quote in the paperSubstantial claim - Interpretationreasoning
The self-reported diagnostic approach introduces considerable bias because it often lacks critical information, such as the timing of the traumatic event, the date of diagnosis, and any treatments received since.
“this diagnostic approach introduces considerable bias, as it often lacks critical information, such as the timing of the traumatic event, the date of diagnosis, and any treatments received since.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 31
- May combine two claims. 39
- The quote only partly states this claim. 1
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSince the late 1990s, there has been growing interest in the use of linguistic markers in psychiatry, with advancements in natural language processing contributing to this development.
“Since the late 1990s, there has been a growing interest in the use of linguistic markers in psychiatry, with advancements in natural language processing (NLP) contributing to this development.[8][9][10]”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [8] Wu, 2023, Systematic review of machine learning in PTSD studies for automated diagnosis evaluation, NPJ Ment Health Res source
The Results (Structured clinical interviews and Self-report questionnaires and narratives; page numbers unavailable) describe the use of speech, lexical and semantic features, including NLP, to assess PTSD. The Discussion describes recent advances in NLP models. The paper does not establish that interest in linguistic markers across psychiatry has grown since the late 1990s or that NLP advances drove that historical trend. (LLM, conf=0.83)
Experimental — accuracy not yet measured - cites [9] Zhang, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [10] Abbe, 2016, Text mining applications in psychiatry: a systematic literature review, Int J Methods Psychiatr Res source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductcitationData extraction was conducted independently by four reviewers following the CHARMS checklist.
“Data extraction was conducted independently by 4 reviewers (R.Q., F.G, Y.A., and S.M.), following the recommendations in the CHARMS checklist.23”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [23] Moons, 2014, Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist, PLoS Med source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationPTSD is a severe mental health condition triggered by a traumatic event and characterized by intrusive memories, avoidance behaviors, negative changes in thought and mood, and alterations in arousal and reactivity.
“Post-traumatic stress disorder (PTSD) is a severe mental health condition triggered by experiencing a traumatic event and characterized by symptoms like intrusive memories, avoidance behaviors, negative changes in thought and mood, and alterations in arousal and reactivity.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [1] American Psychiatric Association source
The supplied passages (11 September 2026, no page given) discuss depression, not PTSD. Searches for PTSD and its proposed symptom clusters returned no passage addressing the claim, so it cannot be verified from our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationOver 80% of PTSD cases involve one or more comorbid disorders such as depression, anxiety, or mild traumatic brain injury.
“A review 5 of the literature revealed that over 80% of PTSD cases involved 1 or more comorbid disorders such as depression, anxiety, or mild traumatic brain injury (TBI).6”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [5] Bryant, 2019, Post-traumatic stress disorder: a state-of-the-art review of evidence and challenges, World Psychiatry source
The PREVALENCE section says epidemiological studies suggest that most people with PTSD have comorbid disorders, particularly depression and anxiety disorders. It does not give an over-80% figure or identify mild traumatic brain injury as an example; it names substance use disorder instead. No page number is available in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [6] Miao, 2018, Posttraumatic stress disorder: from diagnosis to prevention, Mil Med Res source
The “Prevalence and importance” section says that more than 80% of PTSD patients have one or more comorbidities and identifies depression and concurrent mild TBI. It does not identify anxiety as a comorbidity; the “Definition and differential diagnosis” section mentions anxiety disorder only as a condition with similar symptoms to distinguish from PTSD. No page numbers are available in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measured
May combine two claims.
Study design and conductcitationPROBAST was augmented with criteria tailored to machine-learning challenges: interpretability, performance metrics, code availability, and overfitting mitigation.
“To align with best practices 27 for systematic reviews of ML studies, we augmented PROBAST with criteria tailored to these challenges”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [27] Collins, 2024, TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods, BMJ source
The Development of TRIPOD+AI section (page unavailable) describes PROBAST+AI as a tool being developed to assess the quality and risk of bias of machine-learning prediction models. The TRIPOD+AI statement section calls it forthcoming; it does not say PROBAST had already been augmented or identify the four listed criteria as PROBAST+AI criteria. Code, performance, and overfitting appear in the paper’s TRIPOD+AI reporting checklist instead. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [28] Fryer, 2021, Shapley values for feature selection: the good, the bad, and the axioms, IEEE Access source
The Introduction and ‘Feature Selection in General’ sections discuss Shapley-based interpretation and model performance, not systematic-review criteria. Searches for PROBAST and for code availability and overfitting mitigation found no passage about augmenting PROBAST; the cited copy does not address the attributed point. Page numbers were not available in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
InterpretationcitationPresent tense usage often suggests that the traumatic experience is being relived rather than merely remembered, indicating ongoing distress and unresolved trauma.
“Present tense usage often suggests that the traumatic experience is being relived rather than merely remembered (related to criterion B of DSM-5), which can indicate ongoing distress and unresolved trauma.40,77”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
The Methods (Language features; page unavailable) report that prior studies observed increased present-tense use in PTSD narratives, but caution that present tense has several meanings depending on context. The Discussion (page unavailable) links possible literal re-experiencing under Criterion B to lexical features, disfluencies, and perception-related verbs—not specifically to present tense. It does not establish that present-tense use indicates ongoing distress or unresolved trauma. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [77] Hellawell, 2004, A comparison of flashbacks and ordinary autobiographical memories of trauma: content and language, Behav Res Ther source
May combine two claims.
InterpretationcitationFuture tense was less commonly analyzed but has been considered to reflect anticipatory anxiety or fear about potential future threats.
“Meanwhile, the future tense was less commonly analyzed, yet has been considered to reflect anticipatory anxiety or fear about potential future threats, which are significant elements of PTSD symptomatology.40,59”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
The Results associate greater use of future tense with PTSD hypervigilance symptoms, and the Discussion suggests it may reflect altered anticipation of forthcoming activities. The paper does not say that future tense was less commonly analyzed or specifically interpret it as fear or anxiety about future threats. The supplied passages have no page numbers. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [59] Manne, 2002, Language use and post-traumatic stress symptomatology in parents of pediatric cancer survivors, J Appl Soc Pyschol source
May combine two claims.
FindingcitationTrauma narratives with a high density of negative emotion words were associated with poorer psychological adjustment and higher PTSD symptoms.
“Many others [55][56][57][58] also found that trauma narratives with a high density of negative emotion words were associated with poorer psychological adjustment and higher PTSD symptoms.”
Quote in the paperSubstantial claim2 of 3 sources checked- cites [55] Beaudreau, 2007, Are trauma narratives unique and do they predict psychological adjustment?, J Trauma Stress source
- cites [56] Bartal, 2022 source
The Results (Word category analysis; page unavailable) report that women in the provisional childbirth-related PTSD group used more negative-emotion words in their birth narratives than women without childbirth-related PTSD. The paper does not establish a separate association with poorer psychological adjustment, and its finding concerns childbirth narratives and PTSD group status rather than trauma narratives generally or a graded relationship with symptom severity. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [57] Ul Alam, 2020, LAXARY: source
The Related Works section reports more frequent negative-emotion words among depressed Twitter users and notes language differences in PTSD self-narratives. It does not say that a high density of negative-emotion words in trauma narratives was associated with poorer psychological adjustment or higher PTSD symptom levels; the paper's PTSD tweet analysis instead reports greater use of anxiety words. (LLM, conf=0.79)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationcitationAn increase in cognitive processing words could reflect heightened engagement with trauma content, potentially leading to more symptoms at first.
“Conversely, other research 41,48,74 indicated that an increase in cognitive processing words could reflect heightened engagement with trauma content, potentially leading to more symptoms at first.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [41] Fernández-Lansac, 2017, Quality of memories in women abused by their intimate partner: analysis of traumatic and nontraumatic narratives, J. Trauma. Stress source
- cites [48] D’Andrea, 2012, Linguistic predictors of post-traumatic stress disorder symptoms following 11 September 2001, Appl Cogn Psychol source
The Interpretation of data section says greater use of cognitive-mechanism words shortly after 9/11 predicted greater hyperarousal symptoms months later, and suggests those words may reflect attempts to process trauma-related information. It measured greater use, not an increase in word use, and did not find more symptoms initially; it also offers emotional distancing as an alternative interpretation. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [74] Booker, 2018, Narratives in the immediate aftermath of traumatic injury: markers of ongoing depressive and posttraumatic stress disorder symptoms, J Trauma Stress source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataStudy selection was crosschecked with existing literature reviews, and no additional studies were added.
“In order to avoid missing studies, we crosschecked our study selection with existing literature reviews 8,10,13,17,19 and no additional studies were added.”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites [8] Wu, 2023, Systematic review of machine learning in PTSD studies for automated diagnosis evaluation, NPJ Ment Health Res source
- cites [10] Abbe, 2016, Text mining applications in psychiatry: a systematic literature review, Int J Methods Psychiatr Res source
- cites [13] Zhang, 2022, Natural language processing applied to mental illness detection: a narrative review, NPJ Digit Med source
May combine two claims.
PremisecitationMachine-learning models often involve higher complexity, greater risks of overfitting, and distinct interpretability needs compared with traditional regression.
“Unlike traditional regression, ML models often involve higher complexity (eg, non-linear algorithms, hyperparameter tuning), greater risks of overfitting, and distinct interpretability needs.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [26] Moriarty, 2022, Predicting relapse or recurrence of depression: systematic review of prognostic models, Br J Psychiatry source
The ROB and applicability assessment (p. 7) says machine-learning models may require substantially larger samples than regression models to achieve low optimism, and the Implications section (p. 8) identifies lack of interpretability as a criticism. The paper does not establish the claimed higher complexity or give non-linear algorithms and hyperparameter tuning as examples; it also discusses overfitting as a risk for statistical models generally rather than clearly establishing a greater risk for machine learning. (LLM, conf=0.85)
Experimental — accuracy not yet measured
May combine two claims. The quote only partly states this claim.
FindingreasoningStudies with the lowest risk-of-bias (RoB) score fell into two categories: clinical studies focusing on very simple linguistic features (e.g., narrative length or reading scores) or studies using manual annotators that increased the events-per-variable (EPV) number while maintaining gold standards; and studies involving multiple researchers from different departments or computer science teams, and
“The studies with the lowest RoB score fell into 2 categories: clinical studies focusing on very simple linguistic features, such as narrative length 85 or reading scores, 38 or studies using manual annotators, 77 which increased the EPV number while maintaining gold standards for diagnostic tools, participant inclusion, and linguistic features.”
- Also stated in: § Risk of bias · plain
“The other type of study with low RoB involved multiple researchers from different departments to incorporate best practices from all disciplines (using gold standards for diagnosis and study design), along with robust and reliable statistical frameworks and linguistic analysis tools 56,60,86,87 or computer science teams 88 using publicly available datasets.”
- Also stated in: § Summary of findings · hedged
“Studies with the lowest RoB often focused on simple linguistic characteristics or employed multidisciplinary teams using robust statistical methods.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [85] Fernández-Lansac, 2015, Narrative length and speech rate in battered women, PloS One source
- cites [38] Gray, 2001, Complexity of trauma narratives as an index of fragmented memory in PTSD: a critical analysis, Appl Cogn Psychol source
- cites [77] Hellawell, 2004, A comparison of flashbacks and ordinary autobiographical memories of trauma: content and language, Behav Res Ther source
- Also stated in: § Risk of bias · plain
May combine two claims.
FindingcitationOne study applied LIWC to translated data, introducing high risk of bias, while another applied it to a very small text of 50 words.
“One study 67 even applied LIWC to translated data, which introduced a high risk of bias (RoB), while another study 73 applied it to a very small text (50 words).”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [67] Ng, 2015, Narrative characteristics of genocide testimonies predict posttraumatic stress disorder symptoms years later, Psychol Trauma source
The Measures section (p. 5) says this study used LIWC, and Strengths and Limitations (p. 8) says its LIWC measurements depended on translations from Kinyarwanda to English. The authors acknowledge possible measurement error but do not characterize it as a high risk of bias. The 50-word claim is attached to a different citation and is not assessed here. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [73] Marshall, 2020, Prospective analysis of linguistic analysis as a method for assessing trauma symptoms after Hurricane Harvey among Houstonian adults, Traumatology source
May combine two claims.
Findingown dataStudies with notably higher AUC scores include Reece et al, Kirinde Gamaarachchige and Inkpen, and Schultebraucks et al.
“From the forest plot highlights studies with notably higher AUC scores, particularly Reece et al, 52 Kirinde Gamaarachchige and Inkpen, 84 and Schultebraucks et al. 43”
Quote in the paperSubstantial claim2 of 3 sources checked- cites [52] Reece, 2017, Forecasting the onset and course of mental illness with Twitter data, Sci Rep source
The Results (Machine learning classifier; page unavailable) show an ROC curve and report precision, recall, specificity, and F1 scores, but our copy gives no AUC score for the Reece study. It therefore cannot verify the citing document’s characterization of that study as having a notably higher AUC in its forest plot. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [84] Kirinde Gamaarachchige, 2019 source
- cites [43] Schultebraucks, 2020, Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood, Psychol Med source
May combine two claims.
InterpretationreasoningThe first two studies with higher AUC scores leverage social media data and rely on self-reported PTSD diagnoses, which may contribute to their higher AUC values.
“The first 2 studies leverage social media data and rely on selfreported PTSD diagnoses, which may contribute to their higher AUC values.”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [52] Reece, 2017, Forecasting the onset and course of mental illness with Twitter data, Sci Rep source
- cites [84] Kirinde Gamaarachchige, 2019 source
The paper’s Comparison to related work section reports an AUC above 0.90 for its PTSD classification, and the Conclusion identifies social media text as its data source (page numbers unavailable in our copy). The copy does not establish that its PTSD labels came from self-reported diagnoses or suggest that self-reporting contributed to the higher AUC. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationInterpretability tools ensure transparency, a priority emphasized in recent machine-learning reporting guidelines.
“we incorporated the use of interpretability tools (eg, SHAP 28 values, feature importance) to ensure transparency, a priority emphasized in recent ML reporting guidelines.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [28] Fryer, 2021, Shapley values for feature selection: the good, the bad, and the axioms, IEEE Access source
May combine two claims.
FindingreasoningSome studies (42,61,84,88) were unclear about their analysis because they used uninterpretable features or did not use methods like SHAP to suggest interpretations, and they limited their results to classification performance.
“Some 42,61,84,88 were unclear about their analysis because they used uninterpretable features or did not use methods like SHAP to suggest interpretations, and they limited their results to classification performance.”
Quote in the paperNot a substantial claim1 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationFew studies presented an interesting result in the sense of focusing on symptoms such as intrusions and avoidance.
“few studies presented an interesting result in this sense.40,42,54,59,80,90”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [40] Quillivic, 2024, Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning, Sci Rep source
- cites [42] Zhang, 2022 source
- cites [54] Paquet, 2024, Implications of language use in posttraumatic nightmares on psychological symptoms, Psychol Trauma source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationNarrative coherence has been well studied in Formal Thought Disorder research.
“For example, narrative coherence has been well studied in Formal Thought Disorder research.91,92”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [91] Corcoran, 2020, Language as a biomarker for psychosis: a natural language processing approach, Schizophr Res source
- cites [92] Andreasen, 1979, Thought, language, and communication disorders. I. Clinical assessment, definition of terms, and evaluation of their reliability, Arch Gen Psychiatry source
Probably not a substantial claim: may not pass S4 (based).
Findingnone givenThis is the first systematic review investigating the use of language as a marker for PTSD.
“This is the first in the field investigating the use of language as a marker for PTSD.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationOther research suggested that shorter narratives are linked with more severe PTSD.
“Other research has suggested the oppositeshorter narratives are linked with more severe PTSD.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe search resulted in the identification of 593 articles.
“This resulted in the identification of 593 articles”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationLonger trauma narratives were associated with better psychological adjustment.
“Beaudreau 55 observed that longer trauma narratives were associated with better psychological adjustment”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [55] Beaudreau, 2007, Are trauma narratives unique and do they predict psychological adjustment?, J Trauma Stress source
May combine two claims.
Findingown dataFor the overall analysis of 14 studies, Egger's test and the homogeneity test both showed significant results with P-values below .01, indicating significant publication bias and substantial heterogeneity.
“For the overall analysis of 14 studies, Egger's test and the homogeneity test both showed significant results, with P-values below the .01threshold, indicating significant publication bias and substantial heterogeneity.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe Galbraith plot displays heterogeneity between studies, with four points falling outside the 95% CI lines.
“The Galbraith plot (Figure S6) further displays heterogeneity between the studies, with 4 points falling outside the 95% CI lines.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataFor the analysis excluding social media data (8 studies), both Egger's test and the homogeneity test showed P-values below .01, confirming significant publication bias and persistent heterogeneity.
“Similarly, for the analysis excluding social media data (8 studies), both tests also showed P-values below the .01threshold, confirming the presence of significant publication bias and persistent heterogeneity.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe main bias for outcome occurred in studies using self-reported diagnoses, with no control for time since traumatic event, source of trauma, and quality of diagnosis.
“For the outcome (ie, PTSD diagnosis), the main bias occurred in studies using self-reported diagnoses, with no control for time since traumatic event, source of trauma, and no control for the quality of the diagnosis.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe overall pooled AUC for the 14 studies was 0.81 (95% CI 0.75, 0.85), indicating high performance of classification models used to detect PTSD.
“The overall pooled AUC for these studies was 0.81 (95% confidence interval [CI]: 0.75, 0.85), indicating high performances of the classification models used to detect PTSD (Figure 4).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataA total of 8 studies were assigned a low risk of bias rating.
“A total of 8 studies were assigned a low risk of bias rating.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThere has been a notable increase in publications involving computational methods since 2012.
“demonstrating a notable increase in publications involving computational methods since 2012.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataMost studies had an EPV of less than 5, classified as uncertain, and studies with an EPV of less than 3 were classified as high risk.
“Most studies had an EPV of less than 5, which was classified as uncertain, and studies with an EPV of less than 3 were classified as high risk.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenLarger sample sizes often relied on self-reported data from social media.
“with larger sample sizes often relying on selfreported data from social media.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataBoth PTSD prevalence (coefficient = -0.0027, P = .005) and sample size (coefficient = -1.501e-05, P = .023) negatively correlate with AUC.
“The results indicate that both PTSD prevalence (Coefficient ¼ -0.0027, P ¼ .005)and sample size (Coefficient ¼ -1.501e-05, P¼ .023)negatively correlate with AUC”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAnalysis-related biases were common, with low EPV ratios and issues like not reporting statistical power.
“Analysis-related biases were common, with low EPV ratios and issues like not reporting statistical power.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenSelf-reported PTSD diagnoses varied widely and often lacked quality control.
“Self-reported PTSD diagnoses varied widely and often lacked quality control.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe average PTSD prevalence across studies was 47%.
“and the average PTSD prevalence across studies being 47%.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataCommon issues included inadequate control groups and insufficient control over participants' mental health conditions.
“Common issues included inadequate control groups and insufficient control over participants' mental health conditions.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataMost studies employed a combination of descriptive and inferential statistics, with some utilizing ML methods to build and validate models, and a few employed deep learning models.
“Most studies employed a combination of descriptive and inferential statistics, with some utilizing ML methods to build and validate models, and a few employed deep learning models.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone givenMany studies used LIWC without reporting coverage or evaluating its performance on specific corpora, with some applying it to translated data or very small samples, raising bias concerns.
“Many studies used LIWC without reporting coverage or evaluating its performance on specific corpora, with some applying it to translated data or very small samples, raising bias concerns.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenSpeech disorganization and narrative coherence were well-documented but lack standardized measures, making comparison of results difficult.
“Speech disorganization and narrative coherence were well-documented but lack standardized measures, making the comparison of results difficult.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataSelf-administered tools were the most common diagnostic tool.
“Diagnosis tools varied, with self-administered tools being the most common”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataA protocol for this study was published with PROSPERO under registration number CRD42024501094.
“A protocol for this study was published with the International Prospective Register of Systematic Reviews (PROS-PERO), under registration number CRD42024501094”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThree review authors independently reviewed the titles and abstracts of the studies identified by the search strategy.
“Three review authors (R.Q., S.M., F.G.) independently reviewed the titles and abstracts of the studies identified by our search strategy.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationFour reviewers independently assessed risk of bias in included studies using PROBAST.
“Four reviewers (R.Q., S.M., Y.A., and F.G.) independently assessed the risk of bias in the included studies using the Prediction Model Risk of Bias Assessment Tool (PROBAST).24”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24] Wolff, 2019, PROBAST: a tool to assess the risk of bias and applicability of prediction model studies, Ann Intern Med source
May combine two claims.
InterpretationreasoningConflicting results on narrative length and cognitive processing words highlight the complexity of PTSD, whose symptoms and expression vary widely depending on type of exposure, time to traumatic event, and population characteristics.
“These conflicting results (narrative length and cognitive processing words) highlight the complexity of PTSD which symptoms and expression vary widely depending on type of exposure, time to traumatic event, and population characteristics.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationPTSD memory is fragmented, and this fragmentation should be reflected in the narrative.
“many research indeed considered that PTSD memory is fragmented 38,78 (see also criterion D1 of DSM-5: “Inability to recall key features of the trauma”) and this fragmentation should be reflected in the narrative.”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationShorter narratives might reflect avoidance and a struggle to organize thoughts.
“Bisson Desrochers et al 75 noted that shorter narratives might reflect avoidance and a struggle to organize thoughts”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [75] Bisson Desrochers, 2016, Gender and changes in trauma narrative following CBT for PTSD, J Aggress Maltreatment Trauma source
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
PremisecitationMachine-learning studies often require evaluation beyond traditional metrics, which may better capture class imbalance or model generalizability.
“ML studies often require evaluation beyond traditional metrics (eg, AUC-ROC, F1-score), which may better capture class imbalance or model generalizability.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
InterpretationreasoningNarrative coherence generally suggests a structured and logical retelling of events, which is often associated with better psychological adjustment.
“For this reason, narrative coherence generally suggests a structured and logical retelling of events, which is often associated with better psychological adjustment.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
PremisereasoningTechniques like cross-validation, regularization, and independent test sets are critical in machine learning due to algorithmic complexity and data-driven optimization risks.
“Techniques like cross-validation, regularization, and independent test sets are critical in ML due to algorithmic complexity and data-driven optimization risks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenReproducibility is a cornerstone of machine-learning research, and public code sharing is increasingly mandated to address replication crises in computational science.
“Reproducibility is a cornerstone of ML research, and public code sharing is increasingly mandated to address replication crises in computational science.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataLarger datasets and higher PTSD prevalence are associated with reduced classification performance.
“suggesting that larger datasets (more heterogeneous) and higher PTSD prevalence (more balanced dataset) are associated with reduced classification performance.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
PremisecitationStudies using noninterpretative models are deemed less applicable due to limited clinical utility, as model transparency is an essential criterion for clinical use.
“For example, studies using noninterpretative models are deemed less applicable due to limited clinical utility, as model transparency is an essential criterion for clinical use.25”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25] Chandler, 2020, Using machine learning in psychiatry: the need to establish a framework that nurtures trustworthiness, Schizophr Bull source
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningIntegrating linguistic biomarkers into clinical assessments will enable a more nuanced understanding of PTSD symptoms and their severity, particularly by helping to reduce false negatives through the detection of uncontrollable measures such as avoidance symptoms.
“This integration will enable a more nuanced understanding of PTSD symptoms and their severity, particularly by helping to reduce false negatives through the detection of uncontrollable measures such as avoidance symptoms.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningAn overemphasis on diagnosis rather than symptoms overlooked the evolving nature of PTSD definitions.
“an overemphasis on diagnosis rather than symptoms overlooked the evolving nature of PTSD definitions”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningLanguage features can be used to monitor patients' progress and predict treatment outcomes.
“Similarly, language features can be used to monitor patients' progress and predict treatment outcomes.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningHigh scores on noisy data like psychopathological texts suggest over-fitting and/or bias in participant selection.
“Such high scores on noisy data like psychopathological texts suggest over-fitting and/or bias in participant selection.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe use of linguistic biomarkers can significantly improve the accuracy and speed of PTSD diagnoses.
“The use of linguistic biomarkers can significantly improve the accuracy and speed of PTSD diagnoses.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe field is confronted with significant obstacles, including methodological inconsistencies, the lack of validation for existing tools, and biases.
“the field is confronted with significant obstacles, including methodological inconsistencies, the lack of validation for existing tools, and biases.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThe incorporation of linguistic biomarkers into clinical practice has the potential to facilitate more accurate, timely, and nuanced PTSD diagnoses, which will ultimately benefit both patients and clinicians.
“Overall, the incorporation of linguistic biomarkers into clinical practice has the potential to facilitate more accurate, timely, and nuanced PTSD diagnoses, which will ultimately benefit both patients and clinicians.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Despite decades of proof-of-concept studies, no effective clinical decision support system has been developed for PTSD.
“However, despite decades of proof-of-concept studies, no effective CDSS has been developed.”
The paper relies on this without support.
Linguistic signals are inherently noisy and susceptible to influence by gender, education, culture, and socioeconomic background.
“However, linguistic signals are inherently noisy and susceptible to be influenced by several factors including gender, education, culture, and socioeconomic background.”
The paper relies on this without support.
Diagnostic efforts for PTSD are frequently complicated by elevated rates of false negatives and psychiatric comorbidities, which can mask core PTSD symptoms.
“Moreover, diagnostic efforts are frequently complicated by elevated rates of false negatives and the presence of psychiatric comorbidities, which can mask the core PTSD symptoms.”
The paper relies on this without support.
Language provides a promising and readily accessible route to understanding human suffering, making it a valuable tool for identifying PTSD.
“Language provides a promising and readily accessible route to understanding human suffering, which makes it a valuable tool for identifying PTSD.”
The paper relies on this without support.
In the absence of biomarkers for PTSD in the healthcare system, linguistic markers could be used to increase the accuracy and validity of diagnosis.
“In the absence of biomarkers (biological, radiological, and genetic) in the healthcare system for PTSD, linguistic markers could be used to increase the accuracy and validity of diagnosis.”
The paper relies on this without support.
Symptom variation has been directly correlated with time since exposure.
“as symptom variation has been directly correlated with time since exposure.”
The paper relies on this without support.
The concept of 'agency' is associated with the development of PTSD symptoms and severity.
“research has shown that the concept of 'agency' is associated with the development of PTSD symptoms and severity.”
The paper relies on this without support.
Agency can be measured using passive voice, state verbs, and certain pronouns.
“Past linguistic studies have suggested that agency can be measured using passive voice, state verbs, and certain pronouns.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Statistical-method percentages use the wrong denominator
MinorThe paper says descriptive statistics appeared in 51.8% of studies, but 42 of the 58 included studies is 72.4%. The percentages for the other three method categories likewise appear to use the 81 total method instances rather than the number of studies.
Reported 51.8 · recomputed 72.413793
“Descriptive statistics, such as Pearson correlations, ANOVA, and partial correlation analyses (eg, Amir et al 37 ) were the most commonly used, appearing in 51.8% (n ¼ 42) of the studies.”
Comorbidity categories total 59 studies, not 58
MinorThe four reported comorbidity categories sum to 59 studies, although the review includes 58. The final category's stated 3.4% also corresponds to about 2 of 58 studies, not 3.
Reported 58 · recomputed 59
“Regarding comorbidities, 36.2% (n ¼ 21) of the studies included 1 comorbidity, 53.4% (n ¼ 31) reported none, 6.9% (n ¼ 4) included 2 comorbidities, and 3.4% (n ¼ 3) included 3 or more comorbidities.”
Percentage for nine self-report studies is misstated
MinorNine of the 58 included studies is 15.5% to one decimal place, not 15.2%.
Reported 15.2 · recomputed 15.517241
“Self-administered tools were the most frequent (46.6%, n ¼ 27), followed by a clinically administered diagnosis (37.9%, n ¼ 22), and finally, self-report methods accounted for 15.2%, n ¼ 9.”
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 11,883 · see citing works
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · see citing works
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718 · see citing works
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984 · see citing works
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018 · see citing works
Further evaluation of the Dutch supervision system for suicides of mental health care users
(2013) · EUR Research Repository (Erasmus University Rotterdam) · doi:10.1176/appi · Cited by 1,030 · see citing works
Page, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916 · see citing works
Temporal patterns of happiness and information in a global social network: hedonometrics and Twitter
Dodds, P. S., Harris, K. D., Kloumann, I. M., Bliss, C. A., & Danforth, C. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0026752 · Cited by 809 · see citing works
Andreasen, N. C. (1979) · Archives of General Psychiatry · doi:10.1001/archpsyc.1979.01780120045006 · Cited by 746 · see citing works
Machine learning and natural language processing in mental health: systematic review
Le Glaz, A., Haralambous, Y., Kim-Dufor, D.-H., Lenca, P., Billot, R., Ryan, T. C., Marsh, J., DeVylder, J., Walter, M., Berrouiguet, S., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/15708 · Cited by 576 · see citing works
Post‐traumatic stress disorder: a state‐of‐the‐art review of evidence and challenges
Bryant, R. A. (2019) · World Psychiatry · doi:10.1002/wps.20656 · Cited by 528 · see citing works
Change in rape narratives during exposure therapy for posttraumatic stress disorder
Foa, E. B., Molnar, C., & Cashman, L. (1995) · Journal of Traumatic Stress · doi:10.1007/bf02102894 · Cited by 515 · see citing works
Natural language processing applied to mental illness detection: a narrative review
Zhang, T., Schoene, A. M., Ji, S., & Ananiadou, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00589-7 · Cited by 476 · see citing works
Forecasting the onset and course of mental illness with Twitter data
Reece, A. G., Reagan, A. J., Lix, K. L. M., Dodds, P. S., Danforth, C. M., & Langer, E. J. (2017) · Scientific Reports · doi:10.1038/s41598-017-12961-9 · Cited by 401 · see citing works
Shapley values for feature selection: the good, the bad, and the axioms
Fryer, D., Strumke, I., & Nguyen, H. (2021) · IEEE Access · doi:10.1109/access.2021.3119110 · Cited by 347 · see citing works
Measuring post traumatic stress disorder in Twitter
Coppersmith, G., Harman, C., & Dredze, M. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14574 · Cited by 316 · see citing works
What is your research question? An introduction to the PICOT format for clinicians.
(2012) · Europe PMC (PubMed Central) · Cited by 295 · see citing works
Davis, L. L., Schein, J., Cloutier, M., Gagnon-Sanschagrin, P., Maitland, J., Urganus, A., Guerin, A., Lefebvre, P., & Houle, C. R. (2022) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.21m14116 · Cited by 259 · see citing works
Memory and narrative of traumatic events: A literature review.
Crespo, M., & Fernández‐Lansac, V. (2015) · Psychological Trauma Theory Research Practice and Policy · doi:10.1037/tra0000041 · Cited by 195 · see citing works
Language as a biomarker for psychosis: a natural language processing approach
Corcoran, C. M., Mittal, V. A., Bearden, C. E., E. Gur, R., Hitczenko, K., Bilgrami, Z., Savic, A., Cecchi, G. A., & Wolff, P. (2020) · Schizophrenia Research · doi:10.1016/j.schres.2020.04.032 · Cited by 186 · see citing works
A multisite analysis of the fluctuating course of posttraumatic stress disorder
Bryant, R. A., O’Donnell, M. L., Creamer, M., McFarlane, A. C., & Silove, D. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.1137 · Cited by 179 · see citing works
Relationship between trauma narratives and trauma pathology
Amir, N., Stafford, J., Freshman, M. S., & Foa, E. B. (1998) · Journal of Traumatic Stress · doi:10.1023/a:1024415523495 · Cited by 172 · see citing works
Posttraumatic stress disorder: from diagnosis to prevention
Miao, X.-R., Chen, Q.-B., Wei, K., Tao, K.-M., & Lu, Z.-J. (2018) · Military Medical Research · doi:10.1186/s40779-018-0179-0 · Cited by 161 · see citing works
A comparison of flashbacks and ordinary autobiographical memories of trauma: content and language
Hellawell, S. J., & Brewin, C. R. (2004) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(03)00088-3 · Cited by 150 · see citing works
The organisation and content of trauma memories in survivors of road traffic accidents
Jones, C., Harvey, A. G., & Brewin, C. R. (2007) · Behaviour Research and Therapy · doi:10.1016/j.brat.2006.02.004 · Cited by 141 · see citing works
Text mining applications in psychiatry: a systematic literature review
Abbe, A., Grouin, C., Zweigenbaum, P., & Falissard, B. (2016) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1481 · Cited by 130 · see citing works
DLATK: Differential Language Analysis ToolKit
Schwartz, H. A., Giorgi, S., Sap, M., Crutchley, P., Ungar, L., & Eichstaedt, J. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/d17-2010 · Cited by 130 · see citing works
DSM‐5‐TR: overview of what’s new and what’s changed
First, M. B., Yousif, L. H., Clarke, D. E., Wang, P. S., Gogtay, N., & Appelbaum, P. S. (2022) · World Psychiatry · doi:10.1002/wps.20989 · Cited by 130 · see citing works
Pietrzak, R. H., Schechter, C. B., Bromet, E. J., Katz, C. L., Reissman, D. B., Ozbay, F., Sharma, V., Crane, M., Harrison, D., Herbert, R., et al. (2012) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2012.03.011 · Cited by 124 · see citing works
Post-traumatic psychiatric disorders: PTSD is not the only diagnosis
Auxéméry, Y. (2018) · La Presse Médicale · doi:10.1016/j.lpm.2017.12.006 · Cited by 122 · see citing works
Self-citationComplexity of trauma narratives as an index of fragmented memory in PTSD: a critical analysis
Gray, M. J., & Lombardo, T. W. (2001) · Applied Cognitive Psychology · doi:10.1002/acp.840 · Cited by 118 · see citing works
Linguistic predictors of trauma pathology and physical health
Alvarez‐Conrad, J., Zoellner, L. A., & Foa, E. B. (2001) · Applied Cognitive Psychology · doi:10.1002/acp.839 · Cited by 112 · see citing works
Schultebraucks, K., Yadav, V., Shalev, A. Y., Bonanno, G. A., & Galatzer-Levy, I. R. (2022) · Psychological Medicine · doi:10.1017/s0033291720002718 · Cited by 108 · see citing works
The use of machine learning techniques in trauma-related disorders: a systematic review
Ramos-Lima, L. F., Waikamp, V., Antonelli-Salgado, T., Passos, I. C., & Freitas, L. H. M. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2019.12.001 · Cited by 98 · see citing works
He, Q., Veldkamp, B. P., Glas, C. A. W., & de Vries, T. (2017) · Assessment · doi:10.1177/1073191115602551 · Cited by 74 · see citing works
Early Linguistic Markers of Trauma-Specific Processing Predict Post-trauma Adjustment
Kleim, B., Horn, A. B., Kraehenmann, R., Mehl, M. R., & Ehlers, A. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00645 · Cited by 74 · see citing works
Peritraumatic dissociative experiences, trauma narratives, and trauma pathology
Zoellner, L. A., Alvarez‐Conrad, J., & Foa, E. B. (2002) · Journal of Traumatic Stress · doi:10.1023/a:1014383228149 · Cited by 71 · see citing works
Predicting relapse or recurrence of depression: systematic review of prognostic models
Moriarty, A. S., Meader, N., Snell, K. I. E., Riley, R. D., Paton, L. W., Dawson, S., Hendon, J., Chew-Graham, C. A., Gilbody, S., Churchill, R., et al. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.218 · Cited by 71 · see citing works
Trauma narratives: it’s what you say, not how you say it
Jaeger, J., Lindblom, K. M., Parker-Guilbert, K., & Zoellner, L. A. (2014) · Psychological Trauma: Theory, Research, Practice, and Policy · doi:10.1037/a0035239 · Cited by 70 · see citing works
Linguistic features of suicidal thoughts and behaviors: A systematic review
Homan, S., Gabi, M., Klee, N., Bachmann, S., Moser, A.-M., Duri', M., Michel, S., Bertram, A.-M., Maatz, A., Seiler, G., et al. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102161 · Cited by 69 · see citing works
Are trauma narratives unique and do they predict psychological adjustment?
Beaudreau, S. A. (2007) · Journal of Traumatic Stress · doi:10.1002/jts.20206 · Cited by 63 · see citing works
Linguistic predictors of post-traumatic stress disorder symptoms following 11 September 2001
D'Andrea, W., Chiu, P. H., Casas, B. R., & Deldin, P. (2012) · Applied Cognitive Psychology · doi:10.1002/acp.1830 · Cited by 62 · see citing works
Chandler, C., Foltz, P. W., & Elvevåg, B. (2019) · Schizophrenia Bulletin · doi:10.1093/schbul/sbz105 · Cited by 61 · see citing works
Detecting presence of PTSD using sentiment analysis from text data
Sawalha, J., Yousefnezhad, M., Shah, Z., Brown, M. R. G., Greenshaw, A. J., & Greiner, R. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.811392 · Cited by 58 · see citing works
Corona Hernández, H., Corcoran, C., Achim, A. M., de Boer, J. N., Boerma, T., Brederoo, S. G., Cecchi, G. A., Ciampelli, S., Elvevåg, B., Fusaroli, R., et al. (2023) · Schizophrenia Bulletin · doi:10.1093/schbul/sbac215 · Cited by 56 · see citing works
Narrative fragmentation in child sexual abuse: the role of age and post-traumatic stress disorder
Miragoli, S., Camisasca, E., & Di Blasio, P. (2017) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2017.09.028 · Cited by 46 · see citing works
AI and narrative embeddings detect PTSD following childbirth via birth stories
Bartal, A., Jagodnik, K. M., Chan, S. J., & Dekel, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-54242-2 · Cited by 45 · see citing works
Key language markers of depression on social media depend on race
Rai, S., Stade, E. C., Giorgi, S., Francisco, A., Ungar, L., Curtis, B. L., & Guntuku, S. C. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2319837121 · Cited by 45 · see citing works
Bartal, A., Jagodnik, K. M., Chan, S. J., Babu, M. S., & Dekel, S. (2022) · doi:10.1101/2022.08.30.22279394 · Cited by 44 · see citing works
Predicting relapse or recurrence of depression: A systematic review of prognostic models
Moriarty, A. S., Meader, N., Snell, K. I. E., Riley, R. D., Paton, L. W., Dawson, S., Hendon, J., Chew-Graham, C. A., Gilbody, S. M., Churchill, R. C., et al. (2022) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 44 · see citing works
Bartal, A., Jagodnik, K. M., Chan, S. J., Babu, M. S., & Dekel, S. (2023) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2022.100834 · Cited by 44 · see citing works
Son, Y., Clouston, S. A. P., Kotov, R., Eichstaedt, J. C., Bromet, E. J., Luft, B. J., & Schwartz, H. A. (2023) · Psychological Medicine · doi:10.1017/s0033291721002294 · Cited by 42 · see citing works
Trauma narratives and emotional processing
Eid, J., Johnsen, B. H., & Saus, E.-R. (2005) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.2005.00482.x · Cited by 39 · see citing works
Papini, S., Yoon, P., Rubin, M., Lopez-Castro, T., & Hien, D. A. (2015) · Psychological Trauma: Theory, Research, Practice, and Policy · doi:10.1037/tra0000019 · Cited by 37 · see citing works
Linguistic analysis to assess medically related posttraumatic stress symptoms
Shaw, R. J., Harvey, J. E., Nelson, K. L., Gunary, R., Kruk, H., & Steiner, H. (2001) · Psychosomatics · doi:10.1176/appi.psy.42.1.35 · Cited by 31 · see citing works
Systematic review of machine learning in PTSD studies for automated diagnosis evaluation
Wu, Y., Mao, K., Dennett, L., Zhang, Y., & Chen, J. (2023) · npj Mental Health Research · doi:10.1038/s44184-023-00035-w · Cited by 30 · see citing works
Multi-Task, Multi-Channel, Multi-Input Learning for Mental Illness Detection using Social Media Text
Kirinde Gamaarachchige, P., & Inkpen, D. (2019) · Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019) · doi:10.18653/v1/d19-6208 · Cited by 28 · see citing works
Narrative length and speech rate in battered women
Fernández-Lansac, V., & Crespo, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0142651 · Cited by 24 · see citing works
Quillivic, R., Gayraud, F., Auxéméry, Y., Vanni, L., Peschanski, D., Eustache, F., Dayan, J., & Mesmoudi, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-61557-7 · Cited by 24 · see citing works
Self-citationFernández‐Lansac, V., & Crespo, M. (2017) · Journal of Traumatic Stress · doi:10.1002/jts.22154 · Cited by 21 · see citing works
Rivest‐Beauregard, M., Brunet, A., Gaston, L., et al. (2022) · Psychological Assessment · doi:10.1037/pas0001099 · Cited by 16 · see citing works
Linguistic and Non-Linguistic Markers of Disorganization in Psychotic Illness
Liebenthal, E., Ennis, M. M., Rahimi-Eichi, H., Lin, E. C., Chung, Y., & Baker, J. T. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2022.12.003 · Cited by 16 · see citing works
Manne, S. (2002) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2002.tb00233.x · Cited by 15 · see citing works
Auxéméry, Y., & Gayraud, F. (2021) · Annales Médico-psychologiques, revue psychiatrique · doi:10.1016/j.amp.2021.08.015 · Cited by 14 · see citing works
Self-citationNg, L. C., Ahishakiye, N., Miller, D. E., & Meyerowitz, B. E. (2015) · Psychological Trauma: Theory, Research, Practice, and Policy · doi:10.1037/tra0000024 · Cited by 13 · see citing works
Booker, J. A., Graci, M. E., Hudak, L. A., Jovanovic, T., Rothbaum, B. O., Ressler, K. J., Fivush, R., & Stevens, J. (2018) · Journal of Traumatic Stress · doi:10.1002/jts.22271 · Cited by 13 · see citing works
Context- and subgroup-specific language changes in individuals who develop PTSD after trauma
Todorov, G., Mayilvahanan, K., Cain, C., & Cunha, C. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00989 · Cited by 13 · see citing works
Artificial intelligence language predictors of two-year trauma-related outcomes
Oltmanns, J. R., Schwartz, H. A., Ruggero, C., Son, Y., Miao, J., Waszczuk, M., Clouston, S. A. P., Bromet, E. J., Luft, B. J., & Kotov, R. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.09.015 · Cited by 13 · see citing works
Wu, Y., Chen, J., Mao, K., & Zhang, Y. (2023) · 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas58349.2023.10388714 · Cited by 13 · see citing works
Childbirth narratives of women with posttraumatic stress symptoms in the postpartum period
Santoro, E., Stagni-Brenca, E., Olivari, M. G., Confalonieri, E., & Di Blasio, P. (2018) · Journal of Obstetric, Gynecologic & Neonatal Nursing · doi:10.1016/j.jogn.2018.02.009 · Cited by 12 · see citing works
Linguistic predictors of psychological adjustment in healthcare workers during the COVID-19 pandemic
Castiglioni, M., Caldiroli, C. L., Negri, A., Manzoni, G. M., & Procaccia, R. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20054482 · Cited by 12 · see citing works
Auxéméry, Y., & Tarquinio, C. (2020) · Annales Médico-psychologiques revue psychiatrique · doi:10.1016/j.amp.2020.01.004 · Cited by 12 · see citing works
Self-citationAlam, U. (2020) · doi:10.1109/smartcomp50058.2020.00069 · Cited by 11 · see citing works
Filkuková, P., Jensen, T. K., Sofie Hafstad, G., Torvund Minde, H., & Dyb, G. (2016) · European Journal of Psychotraumatology · doi:10.3402/ejpt.v7.29551 · Cited by 10 · see citing works
Gayraud, F., & Auxéméry, Y. (2022) · L'Évolution Psychiatrique · doi:10.1016/j.evopsy.2022.03.005 · Cited by 9 · see citing works
Self-citationFrabetti, M., Gayraud, F., & Auxéméry, Y. (2023) · L'Encéphale · doi:10.1016/j.encep.2022.09.002 · Cited by 9 · see citing works
Self-citationGender and changes in trauma narrative following CBT for PTSD
Bisson Desrochers, A., Beaulieu-Prévost, D., Desautels, J., Békés, V., Belleville, G., Guay, S., & Marchand, A. (2016) · Journal of Aggression, Maltreatment & Trauma · doi:10.1080/10926771.2016.1231147 · Cited by 8 · see citing works
Implications of language use in posttraumatic nightmares on psychological symptoms
Paquet, C. B., Cogan, C. M., & Davis, J. L. (2024) · Psychological Trauma: Theory, Research, Practice, and Policy · doi:10.1037/tra0001266 · Cited by 7 · see citing works
Le développement du discours des personnes blessées psychiques: de l’indicible à la métaphore
Gayraud, F., & Auxéméry, Y. (2023) · Annales Médico-psychologiques, revue psychiatrique · doi:10.1016/j.amp.2021.12.002 · Cited by 7 · see citing works
Self-citationMarshall, K., & Venta, A. (2020) · Traumatology · doi:10.1037/trm0000252 · Cited by 5 · see citing works
Paquet, C., Cogan, C., & Davis, J. (2020) · International Journal of Dream Research · doi:10.11588/ijodr.2020.2.70950 · Cited by 3 · see citing works
Gandelman, E. M., Miller, S. A., & Back, S. E. (2022) · Journal of Traumatic Stress · doi:10.1002/jts.22786 · Cited by 3 · see citing works
Silva, M. A. (2017) · Springer eBooks · doi:10.1007/978-3-319-56782-2_9235-1 · Cited by 3 · see citing works
Speech Behavioral Markers Align on Symptom Factors in Psychological Distress
Zhang, L., Kolacz, J., Rizzo, A., Scherer, S., & Soleymani, M. (2022) · 2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii55700.2022.9953849 · Cited by 2 · see citing works
Key language markers of depression on social media depend on racePreprint
Rai, S., Stade, E. C., Giorgi, S., Francisco, A., Ungar, L., Curtis, B. L., & Guntuku, S. C. (2023) · doi:10.31234/osf.io/uvamz · Cited by 2 · see citing works
The role of embodied cognition in linguistic mental health analysis
(2021) · MODSIM2021, 24th International Congress on Modelling and Simulation. · doi:10.36334/modsim.2021.h2.kernot · Cited by 1 · see citing works
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.07082
Linguistic and Non-Linguistic Markers of Disorganization in Psychotic Illness
Liebenthal, E., Ennis, M., Rahimi-Eichi, H., Lin, E., Chung, Y., & Baker, J. T. (2022) · doi:10.2139/ssrn.4163851
Artificial Intelligence Language Predictors of Two-Year Trauma-Related Outcomes
Oltmanns, J. R., Schwartz, H. A., Ruggero, C., Son, Y., Miao, J., Waszczuk, M., Clouston, S., Bromet, E., Luft, B., & Kotov, R. (2021) · doi:10.31234/osf.io/8fv5r
Flagged first, then by citations
Research works printed but not linked: 3
- Zhang, 2022, A Computational Approach to Understanding Early Trauma and Paediatric PTSD
- Pennebaker, 2007, Linguistic Inquiry and Word Count
- Kernot, 2021, MODSIM Proceedings, 435
Cited by
Sorted by citations
Rubin, M., Stuart, E., Santos, E., Cordova, M. J., & Watters, K. N. (2026) · Journal of Behavioral Medicine · doi:10.1007/s10865-026-00707-w
Marillier, M., Gailhoustet, L., & Abgrall-Barbry, G. (2025) · Annales Médico-psychologiques revue psychiatrique · doi:10.1016/j.amp.2025.10.007
- Authors
- Quillivic, R., Auxéméry, Y., Gayraud, F., Dayan, J., & Mesmoudi, S.
- Affiliations
- École Pratique des Hautes Études (FR)
- Interdisciplinarité en Santé Publique Interventions et Instruments de mesure complexes – Région Est (FR)
- Université Lumière Lyon 2 (FR)
- Neuropsychologie et Imagerie de la Mémoire Humaine (FR)
- Centre Européen de Sociologie et de Science Politique (FR)
- Corresponding author
- Quillivic, Robin
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2025-05-24
- Volume
- 32
- Issue
- 8
- Pages
- 1350-1363
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W4410708503
- Cited by
- 3
- References
- 89
- Citations by year
- 2026: 2 · 2025: 1
- FWCI
- 0.58
- Type
- journal article
- Language
- en
- Open access
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion, other-oa)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Funders
- French General Secretariat for Investment
- National Research Agency
- Programme d’investissement pour l’Avenir (PIA ANR-10-EQPX-0021-01)
- Region île-de-France
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › Posttraumatic Stress Disorder Research
- Health Sciences › Medicine › Epidemiology › Traumatic Brain Injury Research
- Keywords
- post-traumatic stress disorder, linguistic markers, PTSD diagnosis, linguistic analysis, symptom severity, pronoun usage, emotional valence, lexical features, disfluency, risk of bias, clinical utility
- MeSH terms
- Humans, Linguistics, Stress Disorders, Post-Traumatic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review