The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; 12 of the 19 sources its claims cite were checked for support.
AI and narrative embeddings detect PTSD following childbirth via birth stories
Open accessPublished version of a preprint
Abstract
Free-text analysis using machine learning (ML)-based natural language processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated preliminary initial feasibility for this purpose; however, whether it can accurately assess mental illness remains to be determined. This study evaluates the effectiveness of ChatGPT and the text-embedding-ada-002 (ADA) model in detecting post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. Using a sample of 1295 women who gave birth in the last six months and were 18+ years old, recruited through hospital announcements, social media, and professional organizations, we explore ChatGPT's and ADA's potential to screen for CB-PTSD by analyzing maternal childbirth narratives. The PTSD Checklist for DSM-5 (PCL-5; cutoff 31) was used to assess CB-PTSD. By developing an ML model that utilizes numerical vector representation of the ADA model, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.81) ChatGPT and six previously published large text-embedding models trained on mental health or clinical domains data, suggesting that the ADA model can be harnessed to identify CB-PTSD. Our modeling approach could be generalized to assess other mental health disorders.
Overview
CoChat Summary
## TL;DR
This study tested whether childbirth narratives alone could identify probable childbirth-related PTSD (CB-PTSD) using ChatGPT and OpenAI text embeddings. The best model, built on text-embedding-ada-002, performed well and outperformed ChatGPT and six previously published domain-specific PLMs.
## Key findings
- The analyzed sample included 1,295 women who gave birth in the last six months and provided narratives of 30 words or more; narratives were collected on average 2.73 ± 1.82 months postpartum.
- The best model (Model #3, based on text-embedding-ada-002) achieved F1 = 0.81 and AUC = 0.80 for CB-PTSD classification.
- Model #3 achieved 85% sensitivity and 75% specificity in identifying CB-PTSD cases based on narrative language.
- ChatGPT zero-shot and few-shot models performed poorly; the paper reports F1 scores lower than 0.2 for all PLM models in Evaluation Method 1.
## Methods
- Design: Observational study embedded in a larger project on childbirth experience and maternal mental health; participants completed an anonymous web survey and could provide free-text birth stories.
- Sample/data: 1,295 women, age 18+, who had given birth to a live baby in the prior six months; CB-PTSD status was defined using the PCL-5 with cutoff ≥31.
- Analysis: Compared three narrative classifiers: ChatGPT zero-shot (Model #1), few-shot (Model #2), and a custom neural-network classifier using text-embedding-ada-002 embeddings (Model #3). Performance was evaluated with F1 and AUC; the embedding-based model was also compared against six PLMs (all-mpnet-base-v2, mental-roberta-base, mental-bert-base-uncased, mental-xlnet-base-cased, Bio ClinicalBERT, BioGPT).
## Limitations
- The paper states that it did not incorporate additional data sources such as self-report questionnaires beyond PCL-5, medical records, or physiological measures; this limits the model to narrative text alone.
- The sample included women who gave birth before and during the COVID-19 pandemic, and the authors note this heterogeneity may have affected language use and results.
- Clinician evaluations were not performed; CB-PTSD labels relied on the PCL-5 self-report measure. The paper also states that external validation is essential for the DFNN/text-embedding model.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
7 Valid · 1 Not valid · 2 Not checked · 23 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“minoritized women, who are at three times higher risk of experiencing post-traumatic stress symptoms60.”
§ AI and narrative embeddings detect PTSD following childbirth via birth stories > Discussion
What the sources say
- Partly supportedcites [60] Iyengar, A. S. et al. Increased traumatic childbirth and postpartum depression and lack of exclusive breastfeeding in Black and Latinx individuals. Int. J. Gynecol. Obstetr. 158, 759–761 (2022). sourceExperimental — accuracy not yet measured
The Results report that Black and Hispanic/Latinx women who gave birth during the pandemic were three times more likely than matched non-Hispanic White women to report clinically relevant acute traumatic stress to childbirth. The claim broadens that finding to minoritized women generally and calls the measured outcome post-traumatic stress symptoms; the paper says the acute stress level indicates risk for post-traumatic stress disorder, not that it measured those symptoms. (LLM, conf=0.85)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Additionally, Model #3 outperforms previously established models, such as the one presented in our recent previous work47.”
§ AI and narrative embeddings detect PTSD following childbirth via birth stories > Discussion
What the sources say
- Supportedcites [47] Bartal, A., Jagodnik, K. M., Chan, S. J., Babu, M. S. & Dekel, S. Identifying women with post-delivery posttraumatic stress disorder using natural language processing of personal childbirth narratives. Am. J. Obstetr. Gynecol. MFM. 5, 100834 (2023). source
The citation identifies an earlier model, rather than sourcing the citing authors’ comparison with Model #3. The source’s Materials and Methods describes its childbirth-narrative classifier, and its Results reports that model’s performance. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The analyzed sample consists of 1295 women who provided narratives of length 30 words or more”
§ AI and narrative embeddings detect PTSD following childbirth via birth stories > Materials and methods > Study design
plainn = 1,295- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These narratives were collected, on average, 2.73 ± 1.82 months post-childbirth (ranging from 0.02 to 8.34 months).”
- Also stated in: § Discussion · plain
“the childbirth narratives analyzed in our study were collected, on average, 2.73 ± 1.82 months post-childbirth (range: 0.02 to 8.34 months).”
§ AI and narrative embeddings detect PTSD following childbirth via birth stories > Materials and methods > Study design
plainn = 1,295- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Discussion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results of Evaluation Method 2: The results show (Table 4) that our Model #3 with OpenAI’s text-embedding-ada-002 embeddings outperformed Model #3 with other embeddings of PLMs (including embeddings of PLMs trained on clinical or mental health domains) in identifying CB-PTSD using narrative data only.”
§ AI and narrative embeddings detect PTSD following childbirth via birth stories > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
5 more claims
Create a free account to see all 33 claims and every source verdict.
- It may state two separate claims as one. 18
- The quote only partly states this claim. 3
- Probably not a substantial claim. 9
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Set #1: All possible pairs of sentences (C(n, r) = C(1720, 2) = 14,365) in Class 1 (CB-PTSD).”
§ AI and narrative embeddings detect PTSD following childbirth via birth stories > Materials and methods > Narrative analysis
Reported 14365 · recomputed 2956680
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Blevins, C. A., Weathers, F. W., Davis, M. T., Witte, T. K., & Domino, J. L. (2015) · Journal of Traumatic Stress · doi:10.1002/jts.22059 · Cited by 5,233 · cited 2×
“For each participant, we assessed PTSD symptoms associated with childbirth using the Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5) 34,51 , a 20-item self-report measure employed to ascertain the presence and severity of DSM-5 PTSD symptoms after a designated traumatic event over the…”
Christiane, Druml, Wolzt, & Pleiner (2001) · Acta Scientiarum Naturalium Universitatis Sunyatseni · Cited by 3,419
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“Recently, the medical community became intrigued by its capabilities after it demonstrated its proficiency in passing medical board exams 10 , and its potential applications in medical care are wide-ranging 11–13 .”
Bergen, N., & Labonté, R. (2020) · Qualitative Health Research · doi:10.1177/1049732319889354 · Cited by 1,558
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
Wortmann, J. H., Jordan, A. H., Weathers, F. W., Resick, P. A., Dondanville, K. A., Hall-Clark, B., Foa, E. B., Young-McCaughan, S., Yarvis, J. S., Hembree, E. A., et al. (2016) · Psychological Assessment · doi:10.1037/pas0000260 · Cited by 913 · cited 1×
“The PCL-5 is widely recognized for its strong alignment with diagnostic assessments by clinicians and is used to establish a provisional PTSD diagnosis 34,52 (i.e., a presumptive disease state without a formal diagnosis), and is validated in postpartum samples 34 .”
Yildiz, P. D., Ayers, S., & Phillips, L. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.10.009 · Cited by 768 · cited 3×
“In recent years, however, childbirth has become increasingly acknowledged as a significant PTSD trigger 30,33,34 .”
Publicly Available Clinical BERT Embeddings
Alsentzer, E., Murphy, J. R., Boag, W., Weng, W., Jin, D., Naumann, T., & McDermott, M. B. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03323 · Cited by 693
Research works printed but not linked: 2
- Brants, T., Popat, A. C., Xu, P., Och, F. J. & Dean, J. Large language models in machine translation. In Proc. 2007 Joint Conf. Empirical Meth. Nat. Lang. Proc. Computat. Nat. Lang. Learn. Prague. 858–867 (2007).
- Belser, C. A. Comparison of natural language processing models for depression detection in chatbot dialogues. Doctoral dissertation. Massachusetts Institute of Technology. (2023).
Cited by
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244
Large language models for disease diagnosis: a scoping review
Zhou, S., Xu, Z., Zhang, M., Xu, C., Guo, Y., Zhan, Z., Fang, Y., Ding, S., Wang, J., Xu, K., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00011-z · Cited by 243
The Applications of Large Language Models in Mental Health: Scoping Review
Jin, Y., Liu, J., Li, P., Wang, B., Yan, Y., Zhang, H., Ni, C., Wang, J., Li, Y., Bu, Y., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/69284 · Cited by 94
“Therefore, the application of LLMs in mental health is expanding across diverse domains [13-16].”
Hipgrave, L., Goldie, J., Dennis, S. J., & Coleman, A. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1606291 · Cited by 78
Large Language Models in Mental Health Care: A Systematic Scoping Review (Preprint)
hua, Y., Liu, F., Yang, K., Li, Z., Sheu, Y.-H., Zhou, P., Moran, L. V., Ananiadou, S., Clifton, D. A., & Beam, A. (2024) · doi:10.2196/preprints.64088 · Cited by 46
Metadata
- Authors
- Bartal, A., Jagodnik, K. M., Chan, S. J., & Dekel, S.
- Affiliations
- Bar-Ilan University (IL)
- Harvard University (US)
- Massachusetts General Hospital (US)
- Corresponding author
- Dekel, Sharon
- Venue
- Scientific Reports
- Published
- 2024-04-11
- Volume
- 14
- Issue
- 1
- Pages
- 8336-8336
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC11009279
- OpenAlex
- W4394718458
- Cited by
- 45
- References
- 61
- Citations by year
- 2026: 15 · 2025: 25 · 2024: 5
- FWCI
- 21.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Data Science Institute (DSI) at Bar-Ilan University
- Mortimer B. Zuckerman STEM Leadership Program (Project Title: Assessing Risk Factors for Childbirth-Related Post-Traumatic Stress Disorder)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD108619)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Maternal Mental Health During Pregnancy and Postpartum
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- postpartum PTSD, childbirth, ChatGPT, text embeddings, natural language processing, PTSD screening, maternal mental health, birth narratives, PCL-5, narrative classification, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AI and narrative embeddings detect PTSD following childbirth via birth stories. https://app.cochat.ai/papers/10.1038/s41598-024-54242-2
- The paper
- Bartal, A., Jagodnik, K. M., Chan, S. J., & Dekel, S. (2024). AI and narrative embeddings detect PTSD following childbirth via birth stories. Scientific Reports. https://doi.org/10.1038/s41598-024-54242-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.