A Primer in BERTology: What We Know About How BERT Works
Open accessThe DOI matches its Crossref record; 213 of the 213 references it cites were found in a registry or an index.
Abstract
Transformer-based models have pushed state of the art in many areas of NLP, but our understanding of what is behind their success is still limited. This paper is the first survey of over 150 studies of the popular BERT model. We review the current state of knowledge about how BERT works, what kind of information it learns and how it is represented, common modifications to its training objectives and architecture, the overparameterization issue, and approaches to compression. We then outline directions for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“It is advised to apply Bidirectional Encoder Representations from Transformers (BERT) [95,144] to cope with the transferability in cross-domain for cyber grooming detection as it has profoundly bidirectional contextualization and allows the model to gain information from various representations with different positions [146].”
From Online grooming detection: A comprehensive survey of child exploitation in chat logs (2023) · cites this paper as [144]
The cited source says what the claim attributes to it.
In the citing paper
“Contextual models such as BERT 9 can represent each term based on the chat context and lead to a more profound knowledge for semantic analysis [144].”
From Online grooming detection: A comprehensive survey of child exploitation in chat logs (2023) · cites this paper as [144]
The cited source says what the claim attributes to it.
In the citing paper
“Before the emergence of ChatGPT, BERT is a commonly used pre-trained machine learning model for NLP tasks (Wolf et al., 2020), and a ubiquitous baseline in text classification experiments of research in recent years (Rogers et al., 2021).”
From Towards reliable generative AI-driven scaffolding: Reducing hallucinations and enhancing quality in self-regulated learning support (2026) · cites this paper as Rogers et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 5×
“…search-language-understanding-bert 2 Overview of BERT architecture Fundamentally, BERT is a stack of Transformer encoder layers (Vaswani et al., 2017) which consist of multiple “heads”, i.e., fully-connected neural networks augmented with a self-attention mechanism.”
BackgroundInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 5×
“The input representations are computed as fol-Figure 1: BERT fine-tuning (Devlin et al., 2019). lows: BERT first tokenizes the given word into wordpieces (Wu et al., 2016b), and then combines three embedding layers (token, position, and segment) to obtain a fixed-length vector.”
BackgroundMethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“The input representations are computed as fol-Figure 1: BERT fine-tuning (Devlin et al., 2019). lows: BERT first tokenizes the given word into wordpieces (Wu et al., 2016b), and then combines three embedding layers (token, position, and segment) to obtain a fixed-length vector.”
BackgroundMethodInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“Unlike the conventional static embeddings (Mikolov et al., 2013a; Pennington et al., 2014), BERT’s representations are contextualized, i.e., every input token is represented by a vector dependent on the particular context of occurrence.”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 11×
“• Removing NSP does not hurt or slightly improves task performance (Liu et al., 2019b; Joshi et al., 2020; Clinchant et al., 2019), especially in cross-lingual setting (Wang et al., 2019b).”
BackgroundMethodResultInfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“The studies in the knowledge distillation framework (Hinton et al., 2015) use a smaller student-network that is trained to mimic the behavior of a larger teacher-network (BERT-large or BERT-base).”
BackgroundExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“(2020) show the same for open-domain QA using T5 model (Raffel et al., 2019).”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Research works printed but not linked: 76
- Arase Yuki, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 5393
- Baevski Alexei, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 5360
- He Bai, 2020, arXiv:2004. 14996 [cs]
- Bouraoui Zied, 2019, arXiv:1911.12753 [cs] · arXiv
- Brunner Gino, 2020, International Conference on Learning Representations
- Clark Kevin, 2020, International Conference on Learning Representations
- Conneau Alexis, 2019, arXiv:1911.02116 [cs] · arXiv
- Cui Yiming, 2019, arXiv:1906.08101 [cs] · arXiv
- Da Jeff, 2019, Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, 1
- Devlin Jacob, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171
- Elazar Yanai, 2020, arXiv:2006. 00995 [cs]
- Ettinger Allyson, 2019, arXiv: 1907.13528 [cs] · arXiv
- Fan Angela, 2019, International Conference on Learning Representations
- Forbes Maxwell, 2019, Proceedings of the 41st Annual Conference of the Cognitive Science Society (CogSci 2019), 7
- Frankle Jonathan, 2019, International Conference on Learning Representations
- Ganesh Prakhar, 2020, arXiv preprint arXiv:2002.11985 · arXiv
- Garg Siddhant, 2020, AAAI
- Gong Linyuan, 2019, International Conference on Machine Learning, 2337
- Gordon Mitchell A., 2020, arXiv preprint arXiv:2002.08307 · arXiv
- Goyal Saurabh, 2020, arXiv preprint arXiv:2001. 08950
- Hao Yaru, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4143
- Hinton Geoffrey, 2014, Deep Learning and Representation Learning Workshop: NIPS 2014
- Hoover Benjamin, 2019, arXiv:1910. 05276 [cs]
- Jain Sarthak, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 3543
- Jawahar Ganesh, 2019, 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy
- Jiang Haoming, 2019, arXiv preprint arXiv:1911.03437 · arXiv
- Jiang Zhengbao, 2019, arXiv:1911. 12543 [cs]
- Di Jin, 2020, AAAI 2020
- Kim Taeuk, 2020, ICLR 2020
- Kong Lingpeng, 2019, International Conference on Learning Representations
- Kovaleva Olga, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4356
- Krishna Kalpesh, 2020, ICLR 2020
- Kuznetsov Ilia, 2020, arXiv:2004.14999 [cs] · arXiv
- Lan Zhenzhong, 2020, ICLR
- Li Changmao, 2020, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5709
- Lin Yongjie, 2019, Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 241
- Liu Nelson F., 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 1073
- Manning Christopher D., 2020, Proceedings of the National Academy of Sciences, 201907367
- May Chandler, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 622
- Thomas McCoy R., 2019, International Conference on Learning Representations
- Michel Paul, 2019, Advances in Neural Information Processing Systems 32 (NIPS 2019)
- Mikolov Tomas, 2013, Advances in Neural Information Processing Systems 26 (NIPS 2013), 3111
- Jiaqi Mu, 2018, International Conference on Learning Representations
- Pimentel Tiago, 2020, arXiv:2004. 03061 [cs]
- Poerner Nina, 2019, arXiv preprint arXiv: 1911.03681 · arXiv
- Raganato Alessandro, 2020, arXiv:2002.10260 [cs] · arXiv
- Richardson Kyle, 2020, AAAI 2020
- Richardson Kyle, 2019, arXiv:1912.13337 [cs] · arXiv
- Roberts Adam, 2020, arXiv preprint arXiv:2002.08910 · arXiv
- Rogers Anna, 2020, AAAI, 11
- Sanh Victor, 2019, 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS 2019
- Serrano Sofia, 2019, arXiv:1906.03731 [cs] · arXiv
- Shen Sheng, 2019, arXiv preprint arXiv:1909. 05840
- Stickland Asa Cooper, 2019, International Conference on Machine Learning, 5986
- Strubell Emma, 2019, ACL 2019
- Sugawara Saku, 2020, AAAI
- Sun Siqi, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4314
- Talmor Alon, 2019, arXiv:1912.13283 [cs] · arXiv
- Tenney Ian, 2019, International Conference on Learning Representations
- Schijndel Marten van, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 5831
- Vaswani Ashish, 2017, Advances in Neural Information Processing Systems, 5998
- Vig Jesse, 2019, Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 63
- Vilares David, 2020, Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)
- Voita Elena, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4387
- Voita Elena, 2020, arXiv:2003.12298 [cs] · arXiv
- Wallace Eric, 2019, arXiv preprint arXiv:1909. 07940
- Warstadt Alex, 2020, Proceedings of the 42nd Annual Virtual Meeting of the Cognitive Science Society
- Warstadt Alex, 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2870
- Wolf Thomas, 2020, arXiv:1910.03771 [cs] · arXiv
- Felix Wu, 2019, International Conference on Learning Representations
- Zhiyong Wu, 2020, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4166
- Yin Pengcheng, 2020, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8413
- Zadeh Ali Hadi, 2020, arXiv:2005.03842 [cs, stat] · arXiv
- Zhang Zhuosheng, 2020, AAAI 2020
- Zhao Sanqiang, 2019, arXiv preprint arXiv:1909.11687 · arXiv
- Zhou Xuhui, 2020, AAAI 2020
Cited by
Items 601–700 of 1,014
Evaluation of contextual embeddings on less-resourced languagesPreprint
Ulčar, M., Žagar, A., Armendariz, C. S., Repar, A., Pollak, S., Purver, M., & Robnik‐Šikonja, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10614 · Cited by 13
Deciphering Stereotypes in Pre-Trained Language Models
Ma, W., Scheible, H., Wang, B., Veeramachaneni, G., Chowdhary, P., Sun, A., Koulogeorge, A., Wang, L., Yang, D., & Vosoughi, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.697 · Cited by 8
“…for using probing arises from the inherent complexity of Transformer models, which hampers their theoretical analysis, and the widespread acceptance of probing methods as an effective tool to approximate the functioning of these models in natural language understanding (Rogers et al., 2020).”
BackgroundAlper, M., Fiman, M., & Averbuch‐Elor, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00655 · Cited by 26
“It has been observed that language models have an emergent knowledge base property, in which they may be probed for factual knowledge about the world [48, 55].”
BackgroundMethodADVANCEMENTS IN TRANSFORMER ARCHITECTURES FOR LARGE LANGUAGE MODEL: FROM BERT TO GPT-3 AND BEYOND
(2024) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets55985 · Cited by 17
A review on persian question answering systems: from traditional to modern approaches
Jolfaei, S. A., & Mohebi, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11122-z · Cited by 10
“Currently, various QA approaches have been developed, from traditional information retrieval-based systems (Dimitrakis et al. 2019 ; Jurafsky and Martin 2020) to neural approaches (Rogers et al. 2020) and modern LLM-based solutions(Anil et al. 2023; Achiam et al. 2023).”
Mazumder, D., Kumar, A., & Patro, J. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3726866 · Cited by 11
“English-Hindi CM[7] 1661 2914 English 2 2261 3591 Hindi[13] 3338 1416”
Wang, M., Xie, L., Li, C., Wang, X., Sun, M., & Liu, Z. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127631 · Cited by 9
Automated Scoring of Tablet-Administered Expressive Language Tests
Gale, R. C., Bird, J., Wang, Y., van Santen, J. P. H., Prud’hommeaux, E. T., Dolata, J. K., & Asgari, M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.668401 · Cited by 8
“…corpus of unlabeled sentences—3.3 billion words from the English Wikipedia combined with BooksCorpus (Zhu et al., 2015)—language representations produced by BERT have been shown to be effective in capturing both syntactic and semantic aspect of the language (Rogers et al., 2020; Ettinger, 2020).”
BackgroundInfluentialgaBERT — an Irish Language Model
Barry, J., Wagner, J., Cassidy, L., Cowap, A., Lynn, T., Walsh, A., Meachair, M. J. Ó., & Foster, J. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2c485c2jd2yz · Cited by 37
Connecting the Dots in Self-Supervised Learning: A Brief Survey for Beginners
Fang, P., Li, X., Yang, Y., Zhang, S., Kang, Q.-Y., Li, X., & Lan, Z. (2022) · Journal of Computer Science and Technology · doi:10.1007/s11390-022-2158-x · Cited by 11
“Another important component of PTM, network models, is reviewed in [3], where the training objective, model architectures, over-parameterization issue, etc., are thoroughly introduced over BERT-like architectures.”
BackgroundMethodInfluentialDiscontinuous Constituency and BERT: A Case Study of Dutch
Kogkalidis, K., & Wijnholds, G. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.298 · Cited by 9
On Isotropy Calibration of Transformer Models
Ding, Y., Martinkus, K., Pascual, D., Clematide, S., & Wattenhofer, R. P. (2022) · Workshop on Insights from Negative Results in NLP (Insights) · doi:10.18653/v1/2022.insights-1.1 · Cited by 22
Jacobs, C. L., & MacDonald, M. C. (2022) · Cognition · doi:10.1016/j.cognition.2022.105265 · Cited by 13
Enhanced Task-based Knowledge for Lexicon-based Approach in Vietnamese Hate Speech Detection
Hoang, S. N., Nguyen, B. T., Nguyen, N. P., Luu, S. T., Phan, H. T., & Nguyen, H. D. (2022) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse56063.2022.9953615 · Cited by 8
“Besides, various of machine learning approaches and NLP techniques can be adapted for the hate speech detection task such as classification models like Logistic Regression [16], CNN [17], and BERTology [18].”
MethodIntroducing Semantic Information for Numerical Attribute Prediction over Knowledge Graphs
Xue, B., Li, Y., & Zou, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19433-7_1 · Cited by 13
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_2 · Cited by 13
Synonyms, Antonyms and Factual Knowledge in BERT Heads
Serina, L., Putelli, L., Gerevini, A., & Serina, I. (2023) · Future Internet · doi:10.3390/fi15070230 · Cited by 10
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers
Gera, A., Friedman, R., Arviv, O., Gunasekara, C., Sznajder, B., Slonim, N., & Shnarch, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.580 · Cited by 48
A Systematic Search for Compound Semantics in Pretrained BERT Architectures
Miletić, F., & Walde, S. S. I. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.110 · Cited by 24
“…for the whole compound and comparing them against the representations of the individual constituents, often combined using composition functions (Reddy et al., 2011; Schulte im Walde et al., 2013, 2016; Salehi et al., 2014, 2015; Cordeiro et al., 2019; Alipoor and Schulte im Walde, 2020).”
MethodOptimization Chatbot Services Based on DNN-Bert for Mental Health of University Students
Dzaky, A. A., Zeniarja, J., Supriyanto, C., Shidik, G. F., Paramita, C., Subhiyakto, E. R., & Rakasiwi, S. (2024) · Journal of Applied Informatics and Computing · doi:10.30871/jaic.v8i1.7403 · Cited by 8
“Input representation is computed through word tokenization, merging embedding layers, and utilizing specific tokens for classification prediction [21].”
MethodTrevena, W., Zhong, X., Alvarado, M. M., Semenov, A., Oktay, A., Devlin, D., Gohil, A. Y., & Chittimouju, S. H. (2024) · Journal of Medical Internet Research · doi:10.2196/54601 · Cited by 8
Exploring LLM Embedding Potential for Dementia Detection Using Audio Transcripts
Llaca-Sánchez, B. A., García-Noguez, L. R., Aceves-Fernández, M. A., Takács, A., & Tovar‐Arriaga, S. (2025) · Eng—Advances in Engineering · doi:10.3390/eng6070163 · Cited by 13
Yadav, V., Bethard, S. J., & Surdeanu, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.363 · Cited by 17
“(Rogers et al., 2020) on various NLP tasks including QA (van Aken et al.”
BackgroundCan Transformer Language Models Predict Psychometric Properties?
Laverghetta, A., Nighojkar, A., Mirzakhalov, J., & Licato, J. (2021) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2021.starsem-1.2 · Cited by 15
“This sub-category also tests for reasoning over negation, which prior studies found that transformers struggle with (Rogers et al., 2021).”
BackgroundA hierarchical and parallel framework for End-to-End Aspect-based Sentiment Analysis
Xiao, D., Ren, F., Pang, X., Cai, M., Wang, Q., He, M. X., Peng, J., & Fu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.09.021 · Cited by 8
Enriching Language Models with Visually-grounded Word Vectors and the Lancaster Sensorimotor Norms
Kennington, C. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.11 · Cited by 10
“Rogers et al. (2020) provides a recent primer and overview of research that has attempted to uncover strengths and weaknesses of BERT and related language models (so-called BERTology ).”
BackgroundProbing the Role of Positional Information in Vision-Language Models
Rösch, P. J., & Libovický, J. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.77 · Cited by 9
Expressive, Variable, and Controllable Duration Modelling in TTS
Abbas, S. A., Merritt, T., Moinet, A., Karlapati, S., Muszyńska, E., Slangen, S., Gatti, E., & Drugman, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-384 · Cited by 13
ADEPT: Adapter-based Efficient Prompt Tuning Approach for Language Models
Shah, A., Thapa, S., Jain, A., & Huang, L. (2023) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2023.sustainlp-1.8 · Cited by 8
“It stands out to the reason that the lower layers of BERT account for sentence and coarse linguistic structure while the higher layers are more domain-specific and help to learn task-specific parameters (Rogers et al., 2020).”
BackgroundLuu, S. T., Nguyen, K. V., & Nguyen, N. L.-T. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16968-5 · Cited by 8
“With the support of BERT [38], and BERTology [50] on representing texts into features, the task of text classification achieves potential results.”
OoMMix: Out-of-manifold Regularization in Contextual Embedding Space for Text Classification
Lee, S., Lee, D., & Yu, H. (2021) · doi:10.18653/v1/2021.acl-long.49 · Cited by 8
“the underlying structure encoded in its embedding space (Rogers et al., 2021).”
BackgroundDACT-BERT: Differentiable Adaptive Computation Time for an Efficient BERT Inference
Eyzaguirre, C., del Rio, F., Araujo, V., & Soto, Á. (2022) · NLPPOWER · doi:10.18653/v1/2022.nlppower-1.10 · Cited by 9
Conformal Prediction for Text Infilling and Part-of-Speech Prediction
Dey, N., Ding, J., Ferrell, J., Kapper, C., Lovig, M., Planchon, E., & Williams, J. (2022) · The New England Journal of Statistics in Data Science · doi:10.51387/22-nejsds8 · Cited by 23
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework
Zhao, M., Mi, F., Wang, Y., Li, M., Jiang, X., Liu, Q., & Schuetze, H. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.51 · Cited by 18
Improving Sentiment Analysis By Emotion Lexicon Approach on Vietnamese Texts
Doan, A. L., & Luu, S. T. (2022) · International Conference on Asian Language Processing · doi:10.1109/ialp57159.2022.9961318 · Cited by 8
Predicting Fine-Tuning Performance with Probing
Zhu, Z., Shahtalebi, S., & Rudzicz, F. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.793 · Cited by 22
BERTnesia: Investigating the capture and forgetting of knowledge in BERTPreprint
Wallat, J., Singh, J., & Anand, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09313 · Cited by 24
“For a comprehensive survey we point the reader to (Rogers et al., 2020)”
BackgroundResultHuBERT-ECG as a self-supervised foundation model for broad and scalable cardiac applicationsPreprint
Coppola, E., Savardi, M., Massussi, M., Adamo, M., Metra, M., & Signoroni, A. (2024) · medRxiv · doi:10.1101/2024.11.14.24317328 · Cited by 51
ViTHSD: exploiting hatred by targets for hate speech detection on Vietnamese social media texts
Vo, C. N., Huynh, K. B., Luu, S. T., & Do, T.-H. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-024-00348-6 · Cited by 9
“We choose the BERTol-ogy approach [11] because they perform well on the social text according to the benchmark from [12].”
Corbelle, C., Carneiro, V., & Cacheda, F. (2024) · Applied Sciences · doi:10.3390/app14135388 · Cited by 9
“BERT is a cutting-edge natural language processing model developed by Google using a bidirectional transformer architecture to understand the context of words from both directions, enabling deeper language comprehension [19].”
MethodInfusing Multi-Hop Medical Knowledge Into Smaller Language Models for Biomedical Question Answering
Chen, J., Wei, Z., Shen, W., & Shang, R. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3547444 · Cited by 10
Developing Students’ Statistical Expertise Through Writing in the Age of AI
DeLuca, L. S., Reinhart, A., Weinberg, G., Laudenbach, M., Miller, S., & Brown, D. W. (2025) · Journal of Statistics and Data Science Education · doi:10.1080/26939169.2025.2497547 · Cited by 11
“…language models do not explicitly model grammatical structure; while syntactic information can be recovered from properties like token embeddings, there are limits to models’ grammatical sensitivity and “knowledge” (Hewitt and Manning 2019; Htut et al. 2019; Rogers, Kovaleva, and Rumshisky 2021).”
Eyrikaya, E., & Dağ, İ. (2025) · Depression and Anxiety · doi:10.1155/da/9943590 · Cited by 7
“Inlanguage-based tasks, transfer learning is predominantly implemented using large pretrained models, such as BERT [51].”
MBSE Co‐Pilot: A Research Roadmap
Zhang, W., Cockburn, C., Henshaw, M., Douglas, P., Palmer, P. J., Olivier‐Myall, J., & Ji, S. (2025) · Systems Engineering · doi:10.1002/sys.70011 · Cited by 9
“In MBSE, where data can vary significantly between projects and domains, ML models often face difficulty in generalizing effectively across different contexts [50].”
Cross-Temporal Generalization of IndoBERT for Indonesian Hoax News Classification
Riadi, A. T., Indriani, F., Mazdadi, M. I., Faisal, M. R., & Herteno, R. (2025) · Jurnal Teknik Informatika (Jutif) · doi:10.52436/1.jutif.2025.6.5.4757 · Cited by 7
A novel deep neural model for efficient and scalable historical place image classification
Tasir, M. A. M., & Khalid, M. N. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-26897-y · Cited by 8
“The transformer module was configured with 8 multi-head attention units providing optimal attention diversity ablation studies showed 4 heads limited attention diversity for complex architectural features, while 12-16 heads caused attention redundancy and 35% computational overhead increase 58 .”
Teachers’ reading promotion activities: variation, structure and correlates
Ruwette, M., van Schooten, E., & de Glopper, K. (2020) · L1 Educational Studies in Language and Literature · doi:10.17239/l1esll-2020.20.01.09 · Cited by 8
Emerging trends: Deep nets for poets
Church, K., Yuan, X., Guo, S. N., Wu, Z., Yang, Y., & Chen, Z. (2021) · Natural Language Engineering · doi:10.1017/s1351324921000231 · Cited by 7
“Much has been written on “bertology” (Rogers et al. 2020).”
Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations
Taktasheva, E., Mikhailov, V., & Artemova, E. (2021) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2021.mrl-1.17 · Cited by 16
“These studies have proved that LMs are capable of encoding linguistic and hierarchical information (Belinkov and Glass, 2019; Rogers et al., 2020).”
BackgroundTo be or not to be an Integer? Encoding Variables for Mathematical Text
Ferreira, D. M., Thayaparan, M., Valentino, M., Rozanova, J., & Freitas, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.76 · Cited by 7
Emergent Structures and Training Dynamics in Large Language Models
Teehan, R., Clinciu, M., Serikov, O., Szczechla, E., Seelam, N., Mirkin, S., & Gokaslan, A. (2022) · Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models · doi:10.18653/v1/2022.bigscience-1.11 · Cited by 7
“There is no general evidence on how exactly language lev-199 els are distributed across the layers (Rogers et al., 2020).”
BackgroundA Baseline Readability Model for Cebuano
Imperial, J. M., Reyes, L. L. A., Ibanez, M. A., Sapinit, R., & Hussien, M. (2022) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2022.bea-1.5 · Cited by 16
Matsubara, Y., Soldaini, L., Lind, E., & Moschitti, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.537 · Cited by 10
“Our main idea is based on the following considerations: first, in recent years, several transformer models families have been introduced, each pretrained using different datasets and modeling techniques [28].”
BackgroundInfluentialLost in Context? On the Sense-Wise Variance of Contextualized Word Embeddings
Wang, Y., & Zhang, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3337643 · Cited by 10
“C ONTEXTUALIZED word embeddings (CWE) from pre-trained language models [1], [2], [3], [4], [5], [6], [7] have achieved state-of-the-art performance in various downstream tasks.”
BackgroundYusufali, H., Goetze, S., & Moore, R. K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48041-6_29 · Cited by 15
Probing LLMs for Joint Encoding of Linguistic Categories
Starace, G., Papakostas, K., Choenni, R., Panagiotopoulos, A., Rosati, M., Leidinger, A., & Shutova, E. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.476 · Cited by 30
Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number
Hao, S., & Linzen, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.300 · Cited by 19
The ambiguity of BERTology: what do large language models represent?
Gröndahl, T. (2023) · Synthese · doi:10.1007/s11229-023-04435-5 · Cited by 11
“As Rogers et al. (2020, p. 854) put it: “if there is a shortcut in the data, we have no reason to expect BERT to not learn it”.”
InfluentialNguyen, Q. N., Cam, N. T., & Nguyen, K. V. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103814 · Cited by 7
Islakoglu, D. S., Chekol, M. W., & Velegrakis, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60626-7_4 · Cited by 10
Leveraging natural language processing models to automate speech-intelligibility scoring
Herrmann, B. (2024) · Speech Language and Hearing · doi:10.1080/2050571x.2024.2374160 · Cited by 7
“For BERT, the text input is tokenized into ‘wordpieces’ (Devlin et al., 2019; Rogers et al., 2020; Wu et al., 2016).”
InfluentialTowards Efficient Methods in Medical Question Answering using Knowledge Graph Embeddings
Sengupta, S., Heaton, C., Cui, S., Sarkar, S., & Mitra, P. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821824 · Cited by 11
“…however, while ChatGPT has shown remarkable performance on an array of tasks, recent studies [2] suggest that even it struggles with question-answering related tasks across a range of domains which in turn leads us to fall back on BERT-style models [3], [4] which are more capable for MRC [5].”
On the performance of large language models on introductory programming assignments
Raihan, N., Goswami, D., Puspo, S. S. C., Siddiq, M. L., Newman, C. D., Ranasinghe, T., Santos, J. C. S., & Zampieri, M. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00968-y · Cited by 11
“These advance - ments have significantly enhanced performance across various NLP tasks (Rogers et al., 2020 ).”
Transformers, Contextualism, and Polysemy
Grindrod, J. (2026) · Ergo an Open Access Journal of Philosophy · doi:10.3998/ergo.9261 · Cited by 13
Zaman, E. A. K., Isa, N. A. N. M., Jawaddi, S. N. A., & Ismail, A. B. (2026) · Soft Computing · doi:10.1007/s00500-026-11411-x · Cited by 11
Prediction of Time Series Data Based on Transformer with Soft Dynamic Time Wrapping
Ho, K.-H., Huang, P.-S., Wu, I., & Wang, F.-J. (2020) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan49838.2020.9258155 · Cited by 9
“This inspires lots of Transformer-based models, like [5]-[8].”
MethodLook at that! BERT can be easily distracted from paying attention to morphosyntax
Chaves, R. P., & Richter, S. N. (2021) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/b92s-qd21 · Cited by 22
“In this second experiment, we go one step further and probe the ability of BERT and GPT-2 to 3 Although some research suggests attention does not explain BERT’s predictions; see Rogers et al. (2020).”
MethodSame Sentence Prediction: A new Pre-training Task for BERT
Sonmezoz, K., & Amasyalı, M. F. (2021) · Innovations in Intelligent Systems and Applications Conference (ASYU) · doi:10.1109/asyu52992.2021.9598954 · Cited by 7
Text Analysis in Python for Social Scientists: Prediction and Classification
Hovy, D. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108960885 · Cited by 21
Razumovskaia, E., Vulić, I., & Korhonen, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.160 · Cited by 11
“…(Hewitt and Manning, 2019; Jawahar et al., 2019) while (contextual) semantic information is spread across all the layers of a pretrained LM (Tenney et al., 2019), with higher layers capturing increasingly abstract language phenomena (Lin et al., 2019; Rogers et al., 2020; Tenney et al., 2019).”
BackgroundWhat does Chinese BERT learn about syntactic knowledge?
Zheng, J., & Liu, Y. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1478 · Cited by 13
Probing neural language models for understanding of words of estimative probability
Sileo, D., & Moens, M. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.41 · Cited by 17
MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers
Nouriborji, M., Rohanian, O., Kouchaki, S., & Clifton, D. A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.83 · Cited by 12
Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answering
Chaturvedi, A., Bhar, S., Saha, S., Garain, U., & Asher, N. M. (2023) · Computational Linguistics · doi:10.1162/coli_a_00493 · Cited by 16
“One major disadvantage of probing methods is that they fail to address how this information is used during inference (Tenney, Das, and Pavlick 2019; Rogers, Kovaleva, and Rumshisky 2020).”
BackgroundAssessing Grammatical Correctness in Language Learning
Katinskaia, A., & Yangarber, R. (2021) · Työväentutkimus Vuosikirja · Cited by 7
Bird’s Eye: Probing for Linguistic Graph Structures with a Simple Information-Theoretic Approach
Hou, Y., & Sachan, M. (2021) · doi:10.18653/v1/2021.acl-long.145 · Cited by 13
“The results are consistent with existing work (Rogers et al., 2020).”
Linguistic Driven Feature Selection for Text Classification as Stop Word Replacement
Schönle, D., Reich, C., & Abdeslam, D. O. (2023) · Journal of Advances in Information Technology · doi:10.12720/jait.14.4.796-802 · Cited by 9
“Transformers technologies have weaknesses in the explainability of the results [47], multilingualism [48], homogenization [49], data bias [50], and overall understanding the technology [51].”
BackgroundProbing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
Vulić, I., Glavaš, G., Liu, F., Collier, N., Ponti, E. M., & Korhonen, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.153 · Cited by 19
“However, despite the wealth of semantic knowledge stored in the pretrained LMs (Rogers et al., 2020; Vulić et al., 2020b), they do not produce coherent and effective sentence representations when used off-the-shelf (Liu et al.”
BackgroundInspecting the concept knowledge graph encoded by modern language models
Aspillaga, C., Mendoza, M., & Soto, Á. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.263 · Cited by 20
Meta-Learning of Structured Task Distributions in Humans and MachinesPreprint
Kumar, S., Dasgupta, I., Cohen, J. D., Daw, N. D., & Griffiths, T. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02317 · Cited by 12
“…this procedure generates statistically similar but non-compositional distributions, that are in fact easier for down-stream networks to learn than the true compositional distribution—are also relevant to understanding the representations learned by these systems more broadly (Rogers et al., 2020).”
BackgroundDecrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLPPreprint
Rozner, J., Potts, C., & Mahowald, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08620 · Cited by 25
Eigen Analysis of Self-Attention and its Reconstruction from Partial ComputationPreprint
Bhojanapalli, S., Chakrabarti, A., Jain, H., Kumar, S., Łukasik, M., & Veit, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08823 · Cited by 13
“We encourage the reader to see Rogers et al. [16] for an excellent overview of such analyses.”
BackgroundRuleBERT: Teaching Soft Rules to Pre-Trained Language ModelsPreprint
Saeed, M., Ahmadi, N., Nakov, P., & Papotti, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.13006 · Cited by 16
“, 2020) are established tools for capturing both linguistic and factual knowledge (Clark et al., 2019b; Rogers et al., 2020).”
MethodContextualized Sentiment Analysis using Large Language ModelsPreprint
Breitung, C., Kruthof, G., & Müller, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4615038 · Cited by 11
Natural Is the Best: Model-Agnostic Code Simplification for Pre-trained Large Language Models
Wang, Y., Li, X., Nguyen, T. N., Wang, S., Ni, C., & Ding, L. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643753 · Cited by 6
Kauk, J., Kreysa, H., & Schweinberger, S. R. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf028 · Cited by 9
“This was accomplished by combining basic natural language processing (NLP) tools with advanced transformer-based models, which have notably advanced NLP performance recently (cf. (53)).”
Castillo-Campos, M., Becerra‐Alonso, D., & Boomgaarden, H. G. (2025) · Social Science Computer Review · doi:10.1177/08944393251331510 · Cited by 7
“Consequently, numerous studies have emerged, taking advantage of a pre-trained model with ease of scalability (Rogers et al., 2021), allowing for a deeper and more accurate analysis of language, context, and meaning, which is fundamental for media bias detection.”
Predicting the CEFR level of English listening texts with machine learning methods
Cooper, C. R. (2025) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2025.100234 · Cited by 7
A Hybrid Long Short-Term Memory with a Sentiment Analysis System for Stock Market Forecasting
Liagkouras, K., & Metaxiotis, K. (2025) · Electronics · doi:10.3390/electronics14142753 · Cited by 6
Machine Learning and Transformer-based Model for Sentiment Analysis of Indonesian E-Commerce Reviews
Widyananda, W., Maskur, M., & Fauzi, A. (2025) · Indonesian Journal of Computer Science · doi:10.33022/ijcs.v14i4.4980 · Cited by 10
Hwang, T., Seo, H., Jung, J., & Jung, S. (2025) · Applied Sciences · doi:10.3390/app151910434 · Cited by 7
“…responsible for the basic linguistic expressiveness of the language model, while the upper layers tend to learn task-specific representations (Rogers et al., 2020), we hypothesized that this freezing approach would preserve fixed linguistic knowledge while enabling task-specific adaptation.”
Bai, B., Meng, X., & Zhao, C. (2025) · Applied Sciences · doi:10.3390/app151910850 · Cited by 7
Nwaiwu, S., Jongsawat, N., & Tungkasthan, A. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.107783 · Cited by 6
Large Language and Multimodal Models in Archaeological Science: A Review
Qi, X., & Wen, R. (2025) · Electronics · doi:10.3390/electronics14224507 · Cited by 6
Can structural correspondences ground real‐world representational content in large language models?
Williams, I. (2026) · Mind & Language · doi:10.1111/mila.70018 · Cited by 13
“Researchers posit representations of tokens, words, sentences (Pavlick, 2023), distributions over words (Geva et al. 2021), and other syntactic features like roles and parts of speech (Rogers et al., 2020).”
Generative Artificial Intelligence Models: A Survey
Almuammar, M., Alsultan, K., & Altukhaim, Y. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11546-1 · Cited by 9
“Comprehensive overviews of these models are provided in surveys by Rogers et al. ( 2021 ) and Islam and Zhang ( 2024 ).”
Automating Transfer Credit Assessment-A Natural Language Processing-Based Approach
Chandrasekaran, D., & Mago, V. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.027236 · Cited by 9
“(31) conclude in their survey with a clear statement on the limitations of the benchmark datasets, "As with any optimization method, if there is a shortcut in the data, we have no reason to expect BERT to not learn it.”
BackgroundKnowledge-based Intelligent System for IT Incident DevOps
Ahmed, S., Singh, M., Doherty, B., Ramlan, E. I., Harkin, K., Bucholc, M., & Coyle, D. (2023) · doi:10.1109/aiops59134.2023.00005 · Cited by 8
Closing the Gap: Automated Distractor Generation in Japanese Language Testing
Andersson, T., & Picazo‐Sanchez, P. (2023) · Education Sciences · doi:10.3390/educsci13121203 · Cited by 7
Metadata
- Authors
- Rogers, A., Kovaleva, O., & Rumshisky, A.
- Affiliations
- University of Copenhagen (DK)
- University of Massachusetts Lowell (US)
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 842-866
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3006881356
- Cited by
- 1,552
- References
- 182
- Citations by year
- 2026: 171 · 2025: 249 · 2024: 183 · 2023: 292 · 2022: 237 · 2021: 320 · 2020: 100
- FWCI
- 93.20
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1844740)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- BERT, transformer models, pre-trained language models, model interpretability, training objectives, model architecture modification, overparameterization, model compression, learned representations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.