The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Zero-shot interpretable phenotyping of postpartum hemorrhage using large language models
Open accessPublished version of a preprint
Abstract
Many areas of medicine would benefit from deeper, more accurate phenotyping, but there are limited approaches for phenotyping using clinical notes without substantial annotated data. Large language models (LLMs) have demonstrated immense potential to adapt to novel tasks with no additional training by specifying task-specific instructions. Here we report the performance of a publicly available LLM, Flan-T5, in phenotyping patients with postpartum hemorrhage (PPH) using discharge notes from electronic health records ( n = 271,081). The language model achieves strong performance in extracting 24 granular concepts associated with PPH. Identifying these granular concepts accurately allows the development of interpretable, complex phenotypes and subtypes. The Flan-T5 model achieves high fidelity in phenotyping PPH (positive predictive value of 0.95), identifying 47% more patients with this complication compared to the current standard of using claims codes. This LLM pipeline can be used reliably for subtyping PPH and outperforms a claims-based approach on the three most common PPH subtypes associated with uterine atony, abnormal placentation, and obstetric trauma. The advantage of this approach to subtyping is its interpretability, as each concept contributing to the subtype determination can be evaluated. Moreover, as definitions may change over time due to new guidelines, using granular concepts to create complex phenotypes enables prompt and efficient updating of the algorithm. Using this language modelling approach enables rapid phenotyping without the need for any manually annotated training data across multiple clinical use cases.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In healthcare applications, these models have already shown exceptional proficiency in extracting linguistic patterns that convey complex clinical information including clinical phenotypes, assertion status, and relationships between medical concepts (e.g., drug–adverse reaction).12,20, 21, 22, 23, 24”
From irAE-GPT: leveraging large language models to identify immune-related adverse events in electronic health records and clinical trial datasets (2026) · cites this paper as [20]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“For example, Agrawal et al. found that GPT-3 can perform few-shot information extraction on several clinical tasks, including acronym disambiguation, co-reference resolution, and medication extraction 23 .”
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · cited 1×
“Zero-shot natural language processing to extract PPH concepts We utilized Flan-T5, an instruction-tuned LLM that can be applied to unseen tasks without any additional model training 20 , to extract PPH-related concepts from obstetric discharge summaries.”
Practice Bulletin No. 183: postpartum hemorrhage
(2017) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000002351 · Cited by 1,406
Kassebaum, N. J., Barber, R. M., Bhutta, Z. A., Dandona, L., Gething, P. W., Hay, S. I., Kinfu, Y., Larson, H. J., Liang, X., Lim, S. S., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31470-2 · Cited by 1,162
Denny, J. C., Bastarache, L., Ritchie, M. D., Carroll, R. J., Zink, R., Mosley, J. D., Field, J. R., Pulley, J. M., Ramirez, A. H., Bowton, E., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2749 · Cited by 1,013 · cited 1×
“Robust phenotyping is essential to multiple clinical and research work fl ows, such as clinical diagnosis 1 , novel phenotype discovery 2 , clinical trial screening 3 , comparative effectiveness research 4 , quality improvement 5 , and genome-wide and phenome-wide association studies 6 .”
ACOG Practice Bulletin No. 76: Postpartum Hemorrhage
(2006) · Obstetrics & Gynecology · doi:10.1097/00006250-200610000-00046 · Cited by 492
Kuklina, E. V., Whiteman, M. K., Hillis, S. D., Jamieson, D. J., Meikle, S. F., Posner, S. F., & Marchbanks, P. A. (2008) · Maternal and Child Health Journal · doi:10.1007/s10995-007-0256-6 · Cited by 481 · cited 1×
“We leveraged MGB ’ s Research Patient Data Repository (RPDR) 33 to retrieve the data for all MGB patients with a pregnancy related ICD (International Classi fi cation of Diseases) or DRG (Diagnoses Related Groups) code according to the enhanced delivery identi fi cation method 34 .”
PheKB: a catalog and workflow for creating electronic phenotype algorithms for transportability
Kirby, J. C., Speltz, P., Rasmussen, L. V., Basford, M., Gottesman, O., Peissig, P. L., Pacheco, J. A., Tromp, G., Pathak, J., Carrell, D. S., et al. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv202 · Cited by 368
Large language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358 · cited 2×
“This is consistent with prior work that has investigated the utility of GPT-3 in performing few-shot extraction from clinical text 23 .”
Research works printed but not linked: 2
- Aramaki, E., Imai, T., Miyo, K. & Ohe, K. Patient status classification by using rule based sentence extraction and bm25 knn-based classifier. In i2b2 Workshop on Challenges in Natural Language Processing for Clinical Data (2006).
- Lehman, E. et al. Do we still need clinical language models? In: Proceedings of the Conference on Health, Inference, and Learning, PMLR 209, 578–597, (2023).
Cited by
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244
Use of a Large Language Model to Assess Clinical Acuity of Adults in the Emergency Department
Williams, C. Y. K., Zack, T., Miao, B. Y., Sushil, M., Wang, M., Kornblith, A. E., & Butte, A. J. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.8895 · Cited by 148
Retrieval-Augmented Generation–Enabled GPT-4 for Clinical Trial Screening
Ünlü, O., Shin, J., Mailly, C. J., Oates, M., Tucci, M. R., Varugheese, M., Wagholikar, K. B., Wang, F., Scirica, B. M., Blood, A. J., et al. (2024) · NEJM AI · doi:10.1056/aioa2400181 · Cited by 132
Clinical entity augmented retrieval for clinical information extraction
López, I., Swaminathan, A., Vedula, K. S., Narayanan, S., Haredasht, F. N., Ma, S. P., Liang, A. S., Tate, S., Maddali, M. V., Gallo, R. J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01377-1 · Cited by 148
Sushil, M., Zack, T. I., Mandair, D., Zheng, Z., Wali, A., Yu, Y.-N., Quan, Y., Lituiev, D., & Butte, A. J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae146 · Cited by 74
Metadata
- Authors
- Alsentzer, E., Rasmussen, M. J., Fontoura, R., Cull, A. L., Beaulieu-Jones, B., Gray, K. J., Bates, D. W., & Kovacheva, V. P.
- Affiliations
- Brigham and Women's Hospital (US)
- Corresponding author
- Kovacheva, Vesela P.
- Venue
- npj Digital Medicine
- Published
- 2023-11-30
- Volume
- 6
- Issue
- 1
- Pages
- 212-212
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC10689487
- OpenAlex
- W4389205282
- Cited by
- 69
- References
- 35
- Citations by year
- 2026: 16 · 2025: 28 · 2024: 24 · 2023: 1
- FWCI
- 6.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- medRxiv (acceptedVersion)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Chicago (cc-by)
- Funders
- Anesthesia Patient Safety Foundation
- Brigham and Women’s Hospital (IGNITE)
- U.S. Department of Health & Human Services | NIH | National Heart, Lung, and Blood Institute (K08HL161326-01A1, K08 HL146963-02, K08 HL146963-02S1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- postpartum hemorrhage, phenotyping, large language models, Flan-T5, discharge notes, electronic health records, uterine atony, abnormal placentation, obstetric trauma, zero-shot learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Zero-shot interpretable phenotyping of postpartum hemorrhage using large language models. https://app.cochat.ai/papers/10.1038/s41746-023-00957-x
- The paper
- Alsentzer, E., Rasmussen, M. J., Fontoura, R., Cull, A. L., Beaulieu-Jones, B., Gray, K. J., Bates, D. W., & Kovacheva, V. P. (2023). Zero-shot interpretable phenotyping of postpartum hemorrhage using large language models. npj Digital Medicine. https://doi.org/10.1038/s41746-023-00957-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.