Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; 12 of the 19 sources its claims cite were checked for support.

AI and narrative embeddings detect PTSD following childbirth via birth stories

Open access

Published version of a preprint

Abstract

Free-text analysis using machine learning (ML)-based natural language processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated preliminary initial feasibility for this purpose; however, whether it can accurately assess mental illness remains to be determined. This study evaluates the effectiveness of ChatGPT and the text-embedding-ada-002 (ADA) model in detecting post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. Using a sample of 1295 women who gave birth in the last six months and were 18+ years old, recruited through hospital announcements, social media, and professional organizations, we explore ChatGPT's and ADA's potential to screen for CB-PTSD by analyzing maternal childbirth narratives. The PTSD Checklist for DSM-5 (PCL-5; cutoff 31) was used to assess CB-PTSD. By developing an ML model that utilizes numerical vector representation of the ADA model, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.81) ChatGPT and six previously published large text-embedding models trained on mental health or clinical domains data, suggesting that the ADA model can be harnessed to identify CB-PTSD. Our modeling approach could be generalized to assess other mental health disorders.

Overview

CoChat Summary

## TL;DR

This study tested whether childbirth narratives alone could identify probable childbirth-related PTSD (CB-PTSD) using ChatGPT and OpenAI text embeddings. The best model, built on text-embedding-ada-002, performed well and outperformed ChatGPT and six previously published domain-specific PLMs.

## Key findings

  • The analyzed sample included 1,295 women who gave birth in the last six months and provided narratives of 30 words or more; narratives were collected on average 2.73 ± 1.82 months postpartum.
  • The best model (Model #3, based on text-embedding-ada-002) achieved F1 = 0.81 and AUC = 0.80 for CB-PTSD classification.
  • Model #3 achieved 85% sensitivity and 75% specificity in identifying CB-PTSD cases based on narrative language.
  • ChatGPT zero-shot and few-shot models performed poorly; the paper reports F1 scores lower than 0.2 for all PLM models in Evaluation Method 1.

## Methods

  • Design: Observational study embedded in a larger project on childbirth experience and maternal mental health; participants completed an anonymous web survey and could provide free-text birth stories.
  • Sample/data: 1,295 women, age 18+, who had given birth to a live baby in the prior six months; CB-PTSD status was defined using the PCL-5 with cutoff ≥31.
  • Analysis: Compared three narrative classifiers: ChatGPT zero-shot (Model #1), few-shot (Model #2), and a custom neural-network classifier using text-embedding-ada-002 embeddings (Model #3). Performance was evaluated with F1 and AUC; the embedding-based model was also compared against six PLMs (all-mpnet-base-v2, mental-roberta-base, mental-bert-base-uncased, mental-xlnet-base-cased, Bio ClinicalBERT, BioGPT).

## Limitations

  • The paper states that it did not incorporate additional data sources such as self-report questionnaires beyond PCL-5, medical records, or physiological measures; this limits the model to narrative text alone.
  • The sample included women who gave birth before and during the COVID-19 pandemic, and the authors note this heterogeneity may have affected language use and results.
  • Clinician evaluations were not performed; CB-PTSD labels relied on the PCL-5 self-report measure. The paper also states that external validation is essential for the DFNN/text-embedding model.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41598-024-54242-2

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 The three models employed in this study via OpenAI’s API: (1) Model #1 utilizes gpt3.5-turbo-16k for zero-shot classification, (2) Model #2 utilizes gpt3.5-turbo-16k for few-shot learning, and (3) Model #3 utilizes the text-embedding-ada-002 model to train a neural network machine learning (ML) model to screen via classification for childbirth-related post-traumatic stress disorder (CB-PTSD). In the Results section, we also present the results of testing Model #3 with other large text-embedding models.
Figure 1 The three models employed in this study via OpenAI’s API: (1) Model #1 utilizes gpt3.5-turbo-16k for zero-shot classification, (2) Model #2 utilizes gpt3.5-turbo-16k for few-shot learning, and (3) Model #3 utilizes the text-embedding-ada-002 model to train a neural network machine learning (ML) model to screen via classification for childbirth-related post-traumatic stress disorder (CB-PTSD). In the Results section, we also present the results of testing Model #3 with other large text-embedding models.CC BY · PubMed Central
Figure 2 Comparison of Receiver Operating Characteristic (ROC) curves of binary classification Models #1 to #3. The three models directly predict class labels (0 or 1) instead of probabilities. The ‘stepped’ appearance of the ROC curves is due to the models’ binary output, which allows for only 0 or 1 thresholds.
Figure 2 Comparison of Receiver Operating Characteristic (ROC) curves of binary classification Models #1 to #3. The three models directly predict class labels (0 or 1) instead of probabilities. The ‘stepped’ appearance of the ROC curves is due to the models’ binary output, which allows for only 0 or 1 thresholds.CC BY · PubMed Central

Verification

Claims the paper makes

7 Valid · 1 Not valid · 2 Not checked · 23 Own results and interpretations

5 more claims

Create a free account to see all 33 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 18
  • The quote only partly states this claim. 3
  • Probably not a substantial claim. 9

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 61 references

Research works printed but not linked: 2

  1. Brants, T., Popat, A. C., Xu, P., Och, F. J. & Dean, J. Large language models in machine translation. In Proc. 2007 Joint Conf. Empirical Meth. Nat. Lang. Proc. Computat. Nat. Lang. Learn. Prague. 858–867 (2007).
  2. Belser, C. A. Comparison of natural language processing models for depression detection in chatbot dialogues. Doctoral dissertation. Massachusetts Institute of Technology. (2023).

Cited by

See all 45

Metadata

Authors
Bartal, A., Jagodnik, K. M., Chan, S. J., & Dekel, S.
Affiliations
  • Bar-Ilan University (IL)
  • Harvard University (US)
  • Massachusetts General Hospital (US)
Corresponding author
Dekel, Sharon
Venue
Scientific Reports
Published
2024-04-11
Volume
14
Issue
1
Pages
8336-8336
ISSN
2045-2322 (ISSN-L)
OpenAlex
W4394718458
Cited by
45
References
61
Citations by year
2026: 15 · 2025: 25 · 2024: 5
FWCI
21.29
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Data Science Institute (DSI) at Bar-Ilan University
  • Mortimer B. Zuckerman STEM Leadership Program (Project Title: Assessing Risk Factors for Childbirth-Related Post-Traumatic Stress Disorder)
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD108619)
Topics
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Maternal Mental Health During Pregnancy and Postpartum
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
postpartum PTSD, childbirth, ChatGPT, text embeddings, natural language processing, PTSD screening, maternal mental health, birth narratives, PCL-5, narrative classification, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: AI and narrative embeddings detect PTSD following childbirth via birth stories. https://app.cochat.ai/papers/10.1038/s41598-024-54242-2
The paper
Bartal, A., Jagodnik, K. M., Chan, S. J., & Dekel, S. (2024). AI and narrative embeddings detect PTSD following childbirth via birth stories. Scientific Reports. https://doi.org/10.1038/s41598-024-54242-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.