Cited by 54,325Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Open access
Devlin, Jacob, Chang, Ming-Wei, Lee, Kenton, Toutanova, Kristina·2019·Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)

Abstract

Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.

Integrity signals

Registry

Crossref · 10.18653/v1/n19-1423

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Cites

Flagged first, then by citations

2 of the 52 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding https://aclanthology.org/N19-1423/ (Devlin et al., NAACL 2019)
  2. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., NAACL 2019)
  3. ACL materials are Copyright © 1963–2026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.

Cited by

Sorted by citations

Authors
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K.
Venue
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Published
2019
Pages
4171-4186
OpenAlex
W2963341956
Cited by
54,325
References
52
Citations by year
2026: 6191 · 2025: 5078 · 2024: 6786 · 2023: 8262 · 2022: 6552 · 2021: 11198 · 2020: 7387 · 2019: 2821 · 2018: 42 · 2017: 4 · 2013: 1
FWCI
2283.06
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
BERT, bidirectional transformer, pre-training, language comprehension, natural language processing, transformer models, language modeling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review