The DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
Open accessAbstract
Replacing static word embeddings with contextualized word representations has yielded significant improvements on many NLP tasks.However, just how contextual are the contextualized representations produced by models such as ELMo and BERT?Are there infinitely many context-specific representations for each word, or are words essentially assigned one of a finite number of word-sense representations?For one, we find that the contextualized representations of all words are not isotropic in any layer of the contextualizing model.While representations of the same word in different contexts still have a greater cosine similarity than those of two different words, this self-similarity is much lower in upper layers.This suggests that upper layers of contextualizing models produce more context-specific representations, much like how upper layers of LSTMs produce more task-specific representations.In all layers of ELMo, BERT, and GPT-2, on average, less than 5% of the variance in a word's contextualized representations can be explained by a static embedding for that word, providing some justification for the success of contextualized representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Precedent exists for using PCA to extract network-level patterns from high-dimensional neural data, including fMRI (Smith et al., 2015), EEG (Blankertz et al., 2011), and even LLM embeddings (Ethayarajh, 2019).”
From Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis (2025) · cites this paper as Ethayarajh, 2019
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Ethayarajh (2019) notes that unidirectional models produce embeddings that vary smoothly across tokens, implying that the difference operation can still capture meaningful transitions within a span.”
From Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis (2025) · cites this paper as Ethayarajh, 2019
Experimental — accuracy not yet measuredIn the citing paper
“Previous literature showed the BERT embeddings54 capture rich semantic information and are widely recognized as effective for sentence similarity tasks55–57.”
From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913
Neural Word Embedding as Implicit Matrix Factorization
Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,581
Research works printed but not linked: 5
- Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez- Agirre, and Weiwei Guo. 2013. Sem 2013 shared task: Semantic textual similarity, including a pilot on typed-similarity. In SEM 2013: The Second Joint Conference on Lexical and Computational Seman- tics. Association for Computational Linguistics.
- John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word represen- tations. In North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies. Association for Computational Linguistics.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013a. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in Neural Information Processing Systems. pages 3111-3119.
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long Papers). pages 2227-2237.
- Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipan- jan Das, and Ellie Pavlick. 2019. What do you learn from context? probing for sentence structure in contextualized word representations. In Inter- national Conference on Learning Representations. https://openreview.net/forum?id=SJzSgnRcKX.
Cited by
Sorted by citations
Items 701–800 of 898
Multi-Task Unsupervised Question Retrieval Method with Enhanced Keyword FeaturesPreprint
Aiguo, S., & Xinjuan, Z. (2023) · Research Square · doi:10.21203/rs.3.rs-3357612/v1
Optimizacija spletne rešitve v ogrodju Next.js z Google Lighthouse
Pahole, L. (2023) · OTS Sodobne informacijske tehnologije in storitve (OTS) · doi:10.18690/um.feri.8.2023.15
Programiranje v paru z umetno inteligenco
Karakatič, S., & Podgorelec, V. (2023) · OTS Sodobne informacijske tehnologije in storitve (OTS) · doi:10.18690/um.feri.8.2023.1
Lažna božanstva in njih besede
Šprogar, M. (2023) · OTS Sodobne informacijske tehnologije in storitve (OTS) · doi:10.18690/um.feri.8.2023.16
OWASP za vse, ne samo za razvijalce
Gabor, M. (2023) · OTS Sodobne informacijske tehnologije in storitve (OTS) · doi:10.18690/um.feri.8.2023.12
Periti, F., Picascia, S., Montanelli, S., Ferrara, A., & Tahmasebi, N. (2023) · doi:10.36227/techrxiv.24210915 · Cited by 1
Beinborn, L., & Hollenstein, N. (2023) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-43260-6_5
Cluster Validity for Fuzzy Text Segmentation
Lucas, E., & Havens, T. C. (2023) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz52849.2023.10309734
Ding, Y., Jin, R., Paik, J., & Chung, T. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7022-3_35
Gradual Syntactic Label Replacement for Language Model Pre-Training
Wang, Y., Zhang, Y., Li, P., & Liu, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3331096
Which Word Embeddings for Modeling Web Search Queries? Application to the Study of Search Strategies
Ibarboure, C., Tanguy, L., & Amadieu, F. (2023) · International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management · doi:10.5220/0012177600003598
METAPROBE: A Representation- and Task-Agnostic Probe
Zhou, Y., & Srikumar, V. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.18
Ji, Y., Wang, J., Li, J., Ye, H., & Zhang, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.545
Length is a Curse and a Blessing for Document-level Semantics
Xiao, C., Li, Y., Hudson, G., Lin, C., & Moubayed, N. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.86 · Cited by 1
Li, Y., Yin, J., & Lin, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.894
ConRS: Context-response Contrastive Learning for Multi-turn Response Selection
Hu, S., & Zou, Y. (2023) · International Conference on Computer Engineering and Distance Learning (CEDL) · doi:10.1109/cedl60560.2023.00028
Stuhler, O., Stoltz, D. S., & Martin, J. L. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197653609.013.9 · Cited by 1
Corpus Modeling and the Geometries of Text
Stoltz, D. S., Combs, M. A., & Taylor, M. A. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197653609.013.1
TFCSG: An Unsupervised Approach for Question-retrieval Over Multi-task Learning
Aiguo, S., Danner, M., Xinjuan, Z., & Rätsch, M. (2023) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice59929.2023.10354081
Comparison of Transformer-based Architectures for Product Categorization
Mohamed, C., & Hamdy, A. (2023) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca59364.2023.10401734
PACT: Pretraining with Adversarial Contrastive Learning for Text Classification
Khondaker, M. T. I., Abdul-Mageed, M., & Lakshmanan, V. S. L. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.71
Martínez-Mingo, A., Martínez‐Huertas, J. Á., Olmos, R., & Jorge‐Botana, G. (2024) · Science Progress · doi:10.1177/00368504241238773
Semantic Change Detection for the Romanian Language
Truică, C., Tudose, V., & Apostol, E. (2023) · International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc61333.2023.00027
Uncovering patterns of semantic predictability in sentence processingPreprint
Jacobs, C. L., Hubbard, R. J., Federmeier, K. D., & Grobol, L. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/znkpg · Cited by 1
Self-supervised Segment Contrastive Learning for Medical Document Representation
Abro, W. A., Kteich, H., & Bouraoui, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66538-7_31
Kelious, A., & Okirim, M. (2024) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2024.semeval-1.31
News Deja Vu: Connecting Past and Present with Semantic Search
Franklin, B., Silcock, E., Arora, A., Bryan, T., & Dell, M. (2024) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2024.nlpcss-1.8 · Cited by 1
Enhancing recommender systems via data augmentation
Zhang, L. (2023) · doi:10.32657/10356/179298
Ranking Case Law by Context Dimensions Using Fuzzy Fingerprints
Epting, S., Almeida, R., & Carvalho, J. P. (2024) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee60900.2024.10612086
NCSE: Neighbor Contrastive Learning for Unsupervised Sentence Embeddings
Zhang, Z., Lai, P., Wang, R., Ye, F., & Wang, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650907
Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings
Kteich, H., Li, N., Chatterjee, U., Bouraoui, Z., & Schockaert, S. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.86
Castro, L., Temerko, A., & García, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73503-5_19
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.prelim
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.refs
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.index
Standards and resources cited in chapters
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.srcc
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.toc
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.bio
CFP-AL: Combining Model Features and Prediction for Active Learning in Sentence Classification
Kim, K., & Choi, Y. S. (2025) · Applied Sciences · doi:10.3390/app15010482
Qian, L., Zhao, W., Chen, Q., & Chen, J. (2024) · International Conference on Advanced Control Systems and Automation Technologies (ACSAT) · doi:10.1109/acsat63853.2024.10823736
Jiang, Y., Liu, Y., Cui, H., Liu, J., Mao, Y., & He, X. (2025) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2025.3526797
Word Archipelagoes for Explaining Contextual Shifts in Sequential Actions and Events
Ohsawa, Y. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825229
Financial-ICS: Identifying Peer Firms via LongBERT from 10K Reports
Huang, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-0914-7_5
A Maritime Document Knowledge Graph Construction Method Based on Conceptual Proximity Relations
Lin, Y., Yang, T., Shao, Y., Yuan, M., Hu, P., & Li, C. (2025) · Journal of Computer and Communications · doi:10.4236/jcc.2025.132005
Trends in Public Attitudes Toward Government-Mandated Big Data Technology: Evidence from SingaporePreprint
Zhu, L., & Khanal, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5164580
Simple Contrastive Learning with Knowledge Graphs for Story Generation
Zhu, Y., & Pan, R. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889400
Conceptual transformation in distance education: Changes in the period 2006-2023 and future trends
Özçınar, H. (2025) · ODÜ Sosyal Bilimler Araştırmaları Dergisi (ODÜSOBİAD) · doi:10.48146/odusobiad.1613383
Sentimental Awareness Influence on Paraphrase Detection in Transformer Models
Belcher, B. A., Abdulgader, M. M., Osunmakinde, I. O., & Morsi, R. M. B. E. (2024) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca62237.2024.10928018
Amulya, G., Manjunath, G. V. S., Upendra, J., Prasanthi, B. V., Anusha, S., & Madhuri, P. (2025) · International Conference on Intelligent Systems, Advanced Computing and Communication (ISACC) · doi:10.1109/isacc65211.2025.10969255
Word-Aware Modality Stimulation for Multimodal Fusion
Tateishi, S., Osugi, Y., & Nakatsuji, M. (2025) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.40-3_d-o92
Out of Unstructured Data, Atlas! Mapping Strategic Landscapes with Generative AIPreprint
Cheng, Z., Lee, D., Tambe, P. B., & Wu, K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5240714
Multiple feature-anisotropic regularization for out-of-domain intent detection
Wu, D., Wang, X., & Feng, L. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02459-6
Towards data augmentation through geometric distributional extrapolation
Wei, J., & Li, L. (2025) · International Conference on Computer, Artificial Intelligence and Control Engineering (CAICE) · doi:10.1145/3727648.3727744
Prompt based contextualized phrase embeddingPreprint
Tsakam-Sotche, B. (2025) · doi:10.31224/4667
Low-Rank Tensor Encoding Models Decompose Natural Speech Comprehension ProcessesPreprint
Lewis, L., Pitkow, X., & Wehbe, L. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.02.657514
Watermarks for Large Language Models
Chen, P., & Liu, S. (2025) · Springer eBooks · doi:10.1007/978-3-031-76770-8_15
From Anisotropy to Isotropy: The Role of Contrastive Learning in Sentence Representation Learning
Wu, J. F., & Qiu, D. (2024) · International Symposium on Computational Intelligence and Design · doi:10.1109/iscid63852.2024.00032
A Multi-Scale Semantic Textural Similarity Model Based on Discrete Fourier Transform
Zhou, N., & Jin, B. (2025) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit65432.2025.11035807
Bhagavad Gita GPT Unveiling Ancient Wisdom Through Advanced Artificial Intelligence
Kumar, M., Pilania, U., Kumari, S., Marwaha, S., & Gupta, T. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-3253-4_32
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.ind
(2022) · Terminology and lexicography research and practice · doi:10.1075/tlrp.23.ref
Media narratives and the social perception of violence
Bona, M. (2025) · Società e diritti · doi:10.54103/2531-6710/29419
Xu, C., Ding, H., & Zhu, J. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9994-0_35
DPCRec: Dual-Parameterized Contrastive for Multi-Scenario Recommendation
Sun, J., Pan, H., Ye, H. B., Li, R., & Xie, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9901-8_5
Ravikiran, M., Sharma, T., Bhavsar, A. V., & Saluja, R. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99267-4_16
The choice for and the effect of CEO television interviews after earnings announcementsPreprint
Haake, A., & Breuer, W. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5369035
Yang, D., Zhang, X., Han, T., & Liu, Y. (2025) · Natural language processing. · doi:10.1017/nlp.2025.10005 · Cited by 1
Geng, W., & Liang, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5357804
Automated rhetorical move and step recognition in fact-checking articles with neural models
Liu, X., Song, N., Chen, K., Chen, Y., & Pei, L. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104293
Using Next Sentence Prediction for Chinese Frame Disambiguation
Zhang, D., Cao, X., Xue, Y., Lyu, Z., & Wang, R. (2025) · doi:10.1109/icnlp65360.2025.11108503
Martel-Sócala, W., & Salamea, C. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08366-1_30
Enigma Visual: uma abordagem de classificação gamificada de texto
Azeredo, C. S., Farias, K. D. S., de Barros, L. P., Parreiras, M., & Xexeo, G. B. (2025) · Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames) · doi:10.5753/sbgames_estendido.2025.10362
Personalization and Human Context of LLM
Ghodratnama, S. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_54
RKDCSE : Relational Knowledge Distillation For Sentence Representation Learning
Du, W., Li, R., Wu, Y., & Wang, X. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228693
Semantic Representation in Contextual Embeddings: Evidence from Chinese Polysemy
Chen, X. (2025) · Journal of Quantitative Linguistics · doi:10.1080/09296174.2025.2585618
Saifei, L., Niu, J., & Penn, G. (2025) · doi:10.21428/594757db.55d4727f
Generative AI-Based Query Answering System for Construction Safety Regulation
Wang, N. (2025) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784486139.077
Semantically interpretable soft prompt tuning via projection-guided alignment
Liou, C.-S., Wei, C.-R., & Lee, W. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115140
Dynamic Layer-Interaction for Enhanced Sentence Embeddings
Liu, S., Wei, P., & Wang, Y. (2025) · International Conference on Electronic Information Engineering and Computer Science (EIECS) · doi:10.1109/eiecs67708.2025.11283489
Efficient Nearest Neighbor Promptbased Learning for Fewshot Ner in Manufacturing
Chen, J. C., & Wang, P. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.11169726
Exploiting Legal Event Detection Based on the BERT and Graph Convolutional Network
Zhang, Z. (2024) · International Conference on Internet of Things, Robotics and Distributed Computing (ICIRDC) · doi:10.1109/icirdc65564.2024.00130
Exploring Embedding Interpretability by Correspondences Between Topic Models and Text Embeddings
Yuan, M., Rashidi, L., & Zobel, J. (2025) · Information Retrieval Research · doi:10.54195/irrj.23703
Universal Latent Representation in Finite Ring Continuum
Akhtman, Y. (2025) · Entropy · doi:10.3390/e28010040 · Cited by 1
Developing A Question-Answering System In The Uzbek Language Based On The Xlm-Roberta Model
Khujayorov, I., Ochilov, M., & Kholmatov, O. A. (2025) · American Journal Of Applied Science And Technology · doi:10.37547/ajast/volume05issue12-14
Content validity evidence for a personality inventory
Bueno, J. M. H., Primi, R., Cordeiro, E. D. D. A., Jesuíno, A. D. S. A., Muniz, M., & Noronha, A. P. P. (2025) · Psico · doi:10.15448/1980-8623.2025.1.47225
UFORank: Unified Framework of Unsupervised Keyphrase Extraction for Long Documents
Kim, D., & Kang, P. (2026) · IEEE Access · doi:10.1109/access.2026.3654120
Bao, G., Xia, W., Wang, H., Liao, Z., Wu, T., & Zhou, Y. H. (2026) · Sensors · doi:10.3390/s26020656
Li, J., Li, Y., & He, D. (2025) · IEEE International Conference on Civil Aviation Safety and Information Technology (ICCASIT) · doi:10.1109/iccasit66611.2025.11348759
CEVD: Cluster-Based Ensemble Learning for Cross-Project Vulnerability Detection
Cao, Y., Dong, Y., & Liu, J. (2026) · Future Internet · doi:10.3390/fi18020085
Approximating Cognitive Representations Using Space
Starr, J. R., Winship, A., & van Schijndel, M. (2026) · Open Mind · doi:10.1162/opmi.a.389
Detecting and Evaluating Bias in Large Language Models: Concepts, Methods, and Challenges
Gao, Z., Tong, L., & Zhang, Z. J. (2026) · Journal of Behavioral Data Science · doi:10.35566/jbds/gao · Cited by 1
Han, Y. H., & Hur, J. (2026) · Journal of Next-generation Convergence Information Services Technology · doi:10.29056/jncist.2026.02.08
Entropy-Constrained Embeddings for Safe and Reliable Enterprise Large Language Models
Bar, K. (2025) · International Journal of Darshan Institute on Engineering Research and Emerging Technologies · doi:10.32692/ijdi-eret/14.2.2025.2510
Redundancy, Isotropy, and Intrinsic Dimensionalityof Prompt-based Text Embeddings
Tsukagoshi, H., & Sasano, R. (2026) · Journal of Natural Language Processing · doi:10.5715/jnlp.33.76
A Technical Overview of Continuous and Discrete Diffusion-based Language Models
Zhao, H., Huang, J., & Liang, S. (2025) · International Conference on Information Communication and Software Engineering (ICICSE) · doi:10.1109/icicse66971.2025.11430022
Jain, S., Sharma, A., Chodhury, D., Bhowmik, S., & Srivastava, K. (2026) · IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS) · doi:10.1109/sceecs68810.2026.11429845
On Scaling Artifacts in Embedding Post-Processing: A Case Study of the Context Isolation Framework
Yadav, R., Goriparthi, R., & Chaturvedi, R. A. (2026) · Biennial International Conference on Nascent Technologies in Engineering (ICNTE) · doi:10.1109/icnte66387.2026.11437469
Medina, I. F. V., & MURAKAMI, T. (2026) · Journal of Mechanical Design · doi:10.1115/1.4071544
Metadata
- Authors
- Ethayarajh, K.
- Affiliations
- Stanford University (US)
- Corresponding author
- Ethayarajh, Kawin
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 55-65
- OpenAlex
- W2971569798
- Cited by
- 841
- References
- 27
- Citations by year
- 2026: 93 · 2025: 92 · 2024: 115 · 2023: 160 · 2022: 158 · 2021: 157 · 2020: 64
- FWCI
- 36.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- contextualized word representations, word embeddings, BERT, ELMo, GPT-2, embedding geometry, cosine similarity, static embeddings, layer-wise analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
