Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 76 of the 77 references it cites were found in a registry or an index, and 1 are not checked yet.

A large language model for electronic health records

Open access

Abstract

There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records (EHRs). Natural language processing (NLP) powered by pretrained language models is the key technology for medical AI systems utilizing clinical narratives. However, there are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters (compared with billions of parameters in the general domain). It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs. In this study, we develop from scratch a large clinical language model—GatorTron—using >90 billion words of text (including >82 billion words of de-identified clinical text) and systematically evaluate it on five clinical NLP tasks including clinical concept extraction, medical relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA). We examine how (1) scaling up the number of parameters and (2) scaling up the size of the training data could benefit these NLP tasks. GatorTron models scale up the clinical language model from 110 million to 8.9 billion parameters and improve five clinical NLP tasks (e.g., 9.6% and 9.5% improvement in accuracy for NLI and MQA), which can be applied to medical AI systems to improve healthcare delivery. The GatorTron models are publicly available at: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og .

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-022-00742-2

Cites

76 of 77 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Patient distribution by age, gender, race, ethnicity; clinical notes distribution by note type, and clinical department. Ages were calculated as of September 2022.
Fig. 1 Patient distribution by age, gender, race, ethnicity; clinical notes distribution by note type, and clinical department. Ages were calculated as of September 2022.CC BY · PubMed Central
Fig. 2 Training loss and validation loss for GatorTron-base (345 million), medium (3.9 billion), and large (8.9 billion) models. a Training loss. b Validation loss. MLM masked language modeling.
Fig. 2 Training loss and validation loss for GatorTron-base (345 million), medium (3.9 billion), and large (8.9 billion) models. a Training loss. b Validation loss. MLM masked language modeling.CC BY · PubMed Central
Fig. 3 An overview of pretraining and fine-tuning of GatorTron models. We loaded the base model and the medium model into one GPU for distributed training. We sliced the GatorTron-large model into 4 pieces and loaded model pieces to 4 GPUs for distributed training (i.e., model parallelism). TrM transformer unit.
Fig. 3 An overview of pretraining and fine-tuning of GatorTron models. We loaded the base model and the medium model into one GPU for distributed training. We sliced the GatorTron-large model into 4 pieces and loaded model pieces to 4 GPUs for distributed training (i.e., model parallelism). TrM transformer unit.CC BY · PubMed Central
Fig. 4 Pretraining GatorTron-large model with 9 billion parameters using model parallelism. Emb embedding, Tok Token from input sentence, Trm Transformer unit. [SEP]: a token defined in BERT to indicate sentence boundaries. [CLS]: a token defined in BERT for sentence-level representation.
Fig. 4 Pretraining GatorTron-large model with 9 billion parameters using model parallelism. Emb embedding, Tok Token from input sentence, Trm Transformer unit. [SEP]: a token defined in BERT to indicate sentence boundaries. [CLS]: a token defined in BERT for sentence-level representation.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

3 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 73 references

Research works printed but not linked: 2

  1. Yang, X. et al. Identifying relations of medications with adverse drug events using recurrent convolutional neural networks and gradient boosting. J. Am. Med. Inform. Assoc. 27, 65–72 (2020). · DOI · Paper page
  2. Lv, X., Guan, Y., Yang, J. & Wu, J. Clinical relation extraction with deep learning. Int. J. Hybrid. Inf. Technol. 9, 237–248 (2016).

Cited by

See all 1,003

Metadata

Authors
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., Zhang, Y., Magoc, T., Harle, C. A., Lipori, G., Mitchell, D. A., Hogan, W. R., Shenkman, E. A., Bian, J., & Wu, Y.
Affiliations
  • University of Florida (US)
  • Nvidia (United States) (US)
  • University of Florida Health (US)
Corresponding author
Wu, Yonghui
Venue
npj Digital Medicine
Published
2022-12-26
Volume
5
Issue
1
Pages
194-194
ISSN
2398-6352 (ISSN-L)
OpenAlex
W4312220150
Cited by
847
References
73
Citations by year
2026: 171 · 2025: 304 · 2024: 268 · 2023: 101 · 2022: 1
FWCI
55.47
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • IUScholarWorks (Indiana University) (submittedVersion, cc-by)
Funders
  • Patient-Centered Outcomes Research Institute (ME-2018C3-14754)
  • U.S. Department of Health & Human Services | NIH | National Cancer Institute (1R01CA246418)
  • U.S. Department of Health & Human Services | NIH | National Institute on Aging (R56AG069880, R21AG062884)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
electronic health records, clinical natural language processing, clinical concept extraction, semantic textual similarity, natural language inference, medical question answering, model scaling, training data size, clinical text

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A large language model for electronic health records. https://app.cochat.ai/papers/10.1038/s41746-022-00742-2
The paper
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., Zhang, Y., Magoc, T., Harle, C. A., Lipori, G., Mitchell, D. A., Hogan, W. R., Shenkman, E. A., Bian, J., & Wu, Y. (2022). A large language model for electronic health records. npj Digital Medicine. https://doi.org/10.1038/s41746-022-00742-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.