ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
Open accessAbstract
Clinical notes contain information about patients that goes beyond structured data like lab values and medications. However, clinical notes have been underused relative to structured data, because notes are high-dimensional and sparse. This work develops and evaluates representations of clinical notes using bidirectional transformers (ClinicalBERT). ClinicalBERT uncovers high-quality relationships between medical concepts as judged by humans. ClinicalBert outperforms baselines on 30-day hospital readmission prediction using both discharge summaries and the first few days of notes in the intensive care unit. Code and model parameters are available.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Clinical BERT is a more proper domain-specific model trained on all MIMIC-III clinical notes.
“We select Clinical BERT(Huang et al., 2019) as our pre-trained language model since it is a more proper domain-specific model trained on all MIMIC-III clinical notes.”
From A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction (2022) · cites this paper as Huang et al., 2019
BERT variants such as BioBERT and ClinicalBERT improve contextual understanding of medical terminology.
“Variants of Bidirectional Encoder Representations from Transformers (BERT) [5], such as Biomedical BERT (BioBERT) [6] and ClinicalBERT [7], improve the contextual understanding of medical terminology.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [7]
Semantic similarity metrics such as BERTScore and domain-specific variants using ClinicalBERT or SciBERT embeddings assess meaning preservation beyond surface forms.
“Semantic similarity metrics such as BERTScore [132] and domainspecific variants using ClinicalBERT [133] or SciBERT embeddings assess meaning preservation beyond surface forms [65,67,70,90,111].”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [133]
Conventional NLP approaches for temporal information extraction suffer from limited scalability and generalizability and struggle to interpret implicit or vague temporal expressions.
“Conventional natural language processing (NLP) approaches suffer from limited scalability and generalizability, and struggle to interpret implicit or vague temporal expressions.”
From TimeX: Phenotype Onset Extraction from Clinical Narratives (2026) · cites this paper as [21]
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 33 · Cited by 618
- Authors
- Huang, K., Altosaar, J., & Ranganath, R.
- Affiliations
- Harvard University (US)
- Princeton University (US)
- Courant Institute of Mathematical Sciences (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1904.05342
- OpenAlex
- W2937845937
- Cited by
- 618
- References
- 33
- Citations by year
- 2026: 26 · 2025: 97 · 2024: 129 · 2023: 99 · 2022: 74 · 2021: 111 · 2020: 71 · 2019: 11
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- ClinicalBERT, clinical notes, hospital readmission prediction, bidirectional transformer, discharge summaries, clinical text representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
