The DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
Open accessAbstract
Replacing static word embeddings with contextualized word representations has yielded significant improvements on many NLP tasks.However, just how contextual are the contextualized representations produced by models such as ELMo and BERT?Are there infinitely many context-specific representations for each word, or are words essentially assigned one of a finite number of word-sense representations?For one, we find that the contextualized representations of all words are not isotropic in any layer of the contextualizing model.While representations of the same word in different contexts still have a greater cosine similarity than those of two different words, this self-similarity is much lower in upper layers.This suggests that upper layers of contextualizing models produce more context-specific representations, much like how upper layers of LSTMs produce more task-specific representations.In all layers of ELMo, BERT, and GPT-2, on average, less than 5% of the variance in a word's contextualized representations can be explained by a static embedding for that word, providing some justification for the success of contextualized representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Precedent exists for using PCA to extract network-level patterns from high-dimensional neural data, including fMRI (Smith et al., 2015), EEG (Blankertz et al., 2011), and even LLM embeddings (Ethayarajh, 2019).”
From Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis (2025) · cites this paper as Ethayarajh, 2019
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Ethayarajh (2019) notes that unidirectional models produce embeddings that vary smoothly across tokens, implying that the difference operation can still capture meaningful transitions within a span.”
From Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis (2025) · cites this paper as Ethayarajh, 2019
Experimental — accuracy not yet measuredIn the citing paper
“Previous literature showed the BERT embeddings54 capture rich semantic information and are widely recognized as effective for sentence similarity tasks55–57.”
From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913
Neural Word Embedding as Implicit Matrix Factorization
Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,581
Flagged first, then by citations
1 of the 27 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez- Agirre, and Weiwei Guo. 2013. Sem 2013 shared task: Semantic textual similarity, including a pilot on typed-similarity. In SEM 2013: The Second Joint Conference on Lexical and Computational Seman- tics. Association for Computational Linguistics.
- John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word represen- tations. In North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies. Association for Computational Linguistics.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013a. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in Neural Information Processing Systems. pages 3111-3119.
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long Papers). pages 2227-2237.
- Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipan- jan Das, and Ellie Pavlick. 2019. What do you learn from context? probing for sentence structure in contextualized word representations. In Inter- national Conference on Learning Representations. https://openreview.net/forum?id=SJzSgnRcKX.
Cited by
Sorted by citations
Items 801–898 of 898
On the Status of Word Embeddings as Implementations of the Distributional Hypothesis
Mickus, T. (2022) · theses.fr (ABES) · doi:10.70675/cfc2b3e0zdc1dz4f02za2cezd8116a4bd01e
Word Meaning Representation in Neural Language Models : Lexical Polysemy and Semantic Relationships
Soler, A. G. (2021) · theses.fr (ABES) · doi:10.70675/99a10411zc05ez4135z8f31z13573e0fd4d1
Boualili, L. (2022) · theses.fr (ABES) · doi:10.70675/29d19654z89d4z4f44z85efz0add721cac69
Towards Effective, Efficient and Explainable Neural Information Retrieval
Formal, T. (2023) · theses.fr (ABES) · doi:10.70675/0e6d7bcbz4ddbz45bczb909z78ab26702a56
Machine Learning Approaches for Sub-surface Geological Heterogeneous Sources
Arman, M. (2023) · theses.fr (ABES) · doi:10.70675/42bb8288za220z4273za3f7zb042954f2cac
Improving Representations for Language Modeling
Godey, N. (2024) · theses.fr (ABES) · doi:10.70675/b6a15d0ez15bez43e5za967z16a069e0a8bb
Nguyen, D. H. (2024) · theses.fr (ABES) · doi:10.70675/97b1b150zc009z4f96z9ec6z2790870387a0
Bouda, R. G. E., Tapsoba, A. R., Séré, A., & Ouedraogo, F. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11468254
Text-enhanced cross-modal reinforced fusion network for multimodal sentiment analysis
Zhao, W., & Wu, Y. (2026) · Pattern Analysis and Applications · doi:10.1007/s10044-026-01676-1
Crystallizing Semantics: Mapping the Journey of Word Meaning in Language Models
Dwivedi, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-24350-8_16
RELATE: Generating a linguistically inspired Knowledge Graph for fine-grained emotion classification
Schoene, A. M., Dethlefs, N., & Ananiadou, S. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2uz8zds4txxh · Cited by 1
Understanding the Visual Projection Space of Multimodal LLMs
Jeong, S., Song, Y., & Kim, H. J. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00585
Hughes, C., & Hmurovic, J. (2026) · Journal of Marketing · doi:10.1177/00222429261450108
Ghojogh, B., & Ghodsi, A. (2026) · Springer eBooks · doi:10.1007/978-3-032-10738-1_11
Calibrating Stereotype Preference with Multi-Agent Graph Reasoning in Large Language Models
Amin, M. N., & Jesser, A. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491743
Yan, Q., Li, C., Chen, Z., & Wei, F. (2025) · International Conference on Human-Engaged Computing (ICHEC) · doi:10.1145/3786995.3787018
Artificial Intelligence in Studying Russian and Chinese Space Terms
Goncharova, O. V., Митрофанова, И. Ю., & Wang, Y. (2026) · Ethical economy · doi:10.1007/978-3-032-23423-0_23 · Cited by 1
Educational Implications of Lexicographic Misalignment
Proroković, J. (2026) · Magister. · doi:10.5507/magister.2025.514
Can machines imagine? Critical thinking and cultural reasoning in multimodal-multilingual AI
AlAfnan, M. A., MohdZuki, S. F., & AlAfnan, S. M. (2026) · International Journal of Informatics and Communication Technology (IJ-ICT) · doi:10.11591/ijict.v15i2.pp823-838
Ge, Y., Gu, J., & Huang, C. (2026) · Data · doi:10.3390/data11060126
Towards Automating Articles Screening Processes Using Chain-of-Thought Large Language Models
Arpini, C., Cesarini, M., & Lotano, E. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795601
Improving Relative Representations with Learned Anchors and Whitened Inner Products
Svendsen, O. T., Jakobsen, N. H., Mager, F., & Nassar, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38404-1_27
GRAVITAS: A Dynamic Semantic Geometry for Information Retrieval -- Theoretical StudyPreprint
Chraiti, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6866810
Semantic Space of Eurysemic Verbs: Vector Representations
Ефимова, Е. В. (2026) · Филология научные исследования · doi:10.7256/2454-0749.2026.5.79382
Yu, R., Artemev, D., & Yang, C. (2026) · International Conference on Big Data and Informatization Education (ICBDIE) · doi:10.1145/3806980.3807114
Hint-oriented self-suggestive learning for commonsense information processing
Fan, Y., Zou, B., Li, Y., & Hong, Y. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104964
Fine-Grained Prototype-Augmented Named Entity Recognition
Xing, Y., Zhang, X., Wang, P., Ma, Z., Chen, L., Xie, Q., & He, T. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-7996-9_3
Samo, A., & Highhouse, S. (2026) · Journal of Business and Psychology · doi:10.1007/s10869-026-10118-y
Dahlberg, A., Tapiola, O., Luisto, R., Puranen, T., Sanmark, E., & Vartiainen, V. A. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106563
When Textification Fails: Embedding Collapse in Student Dropout Prediction
Aboukacem, A., Bergou, E. H., Berrada, I., Iraqi, Y., & Mekouar, L. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-29788-4_53
Do Geometric Outliers Identify Important Genes in Single-Cell Foundation Models?Preprint
Whalley, J. P. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.22.733850
Hamaguchi, K., Matsuda, Y., & Kuroiwa, J. (2026) · Nonlinear Theory and Its Applications IEICE · doi:10.1587/nolta.17.805
Reading Between the Lines: A Text-based Deep Learning Approach for Understanding Company Dynamics
Duan, H., Yang, Y., & Tam, K. Y. (2026) · MIS Quarterly · doi:10.25300/misq/2026/19345
Martínez-Cruz, R., López-López, Á. J., Portela, J., & Zayat, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7247158
Two Context Effects on Metaphor Usage. A Computational, Corpus-Based Approach
Beekhuizen, B., & Raihert, C. (2026) · Metaphor and Symbol · doi:10.1080/10926488.2026.2627594
REZE: Representation Regularization for Domain-adaptive Text Embedding Pre-finetuning
Lee, S., Lee, J., Lim, H., Kim, S., & Sung, M. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.810
GeoLAN: Geometric Learning of Latent Explanatory Directions in Large Language Models
Pan, T., & Woodard, D. L. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.271
LLMs Explain’t: A Post-Mortem on Semantic Interpretability in Transformer Models
Abdelhalim, A., Edinger, J., Laue, S., & Regneri, M. (2026) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops68308.2026.11585476
53A survey on latent linguistic patterns in bidirectional transformer-based language models
Kugler, K., & Rettinger, A. (2026) · De Gruyter eBooks · doi:10.1515/9783111199740-003
A topic-driven local interpretable model for explainable rumor detection on social media
Pattanaik, B., Mandal, S., Tripathy, R. M., Prasad, D. K., & Sekh, A. A. (2026) · Discover Computing · doi:10.1007/s10791-026-10330-7
García-Hidalgo, C. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02107-7
Representation Analysis of croBERT for Croatian Disinformation Detection
Ljubi, I., & Vukovic, M. (2026) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro70003.2026.11591990
Liu, F. T., Yu, Z., Hu, K., & Wei, R. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2862-1_16
Clustering-Based Methods for Vector-Based Pseudo-Relevance Feedback
Velez, X., Yates, A., Yang, E., Adriaanse, T. V., & Khudanpur, S. P. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809919
INTERPRETATION OF BERT INTERNAL REPRESENTATIONS FOR THE CLASSIFICATION OF TEXTS CONTAINING REASONING
Kanygin, A. (2026) · Mathematical Physics and Computer Simulation · doi:10.15688/mpcm.jvolsu.2026.1.4
Leveraging Large Language Models for Understanding Fundamental Principles of Catalysis
Michtavy, S. S., Perera, S. M. H. D., & Porosoff, M. D. (2026) · The Journal of Physical Chemistry C · doi:10.1021/acs.jpcc.6c02607
Denvis, O. (2026) · Research Square · doi:10.21203/rs.3.rs-10434032/v1
TextBFGS: A Case-Based Reasoning Approach to Code Optimization via Error-Operator Retrieval
Zhang, Z., Liao, Y., Chen, C., He, J., Wu, D., Yu, Q., Gao, Y., Yang, J., Zhang, K., Zhong, X., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-33865-5_19
Rosen, Z. P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7232904
Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings
Wu, S., Chen, Z., Liu, Y., Cui, H., Li, C., & Yan, R. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818204
Chen, L. (2026) · Research Square · doi:10.21203/rs.3.rs-10598944/v1
Do Retrieval-Trained Embeddings Help Linear Contextual Bandits?
Canim, M. (2026) · Mathematics · doi:10.3390/math14162874
Spectral Structure and Semantic Aggregation of Embedding Sets
Rubinshtein, I. A., Zadeba, M., Toropygin, A., Bikmatov, D., Karpasov, I., Nardid, A. N., & Gapanyuk, Y. (2026) · International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd69640.2026.11637313
Context-Sensitive N-Gram Word Partitioning for Improving the Quality of Turkish Word Embeddings
Agun, H. V. (2026) · Applied Sciences · doi:10.3390/app16168184
Predicting the Memorability of Brand Slogans
Aka, A., & McCoy, J. (2026) · Journal of Marketing Research · doi:10.1177/00222437261481110
Moral, A., Antonio Arroyo, S., Aparicio, J., & Barber, X. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8090256
Quantifying Semantic Stability in AI-Mediated Emotional Support Environments
Ferrari, D., Iuliano, A., & Felicetti, F. (2026) · IEEE International Workshop on Metrology for Living Environment (MetroLivEnv) · doi:10.1109/metrolivenv70468.2026.11659862
Mondal, S. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12332-4
Automated Disease Prediction and Prescription System with Regional Kannada Synonym IntegrationPreprint
Gouda, G., & Suma, R. (2026) · Research Square · doi:10.21203/rs.3.rs-10827458/v1
AI-Powered Legal Charge Predictor Evidence Highlighter
Meenakshi, K., Yadav, M., Kushwaha, P. K., & Singh, D. (2026) · International Conference on Trends in Quantum Computing and Emerging Business Technologies (TQCEBT) · doi:10.1109/tqcebt67648.2026.11681036
ID Is Not All You Need: Multimodal Recommendation Without ID Embeddings
Zhang, R., Zhang, Q., & Hu, Z. (2026) · International Conference on Machine Learning and Intelligent Computing · doi:10.1145/3829441.3829566
Aleb, N. (2026) · International journal of intelligent engineering and systems · doi:10.22266/ijies2026.1031.11
Zhang, Y., Ruan, X., Cheng, X., Dong, Z., & Chen, J. (2026) · Evaluation Review · doi:10.1177/0193841x261491411
Scroggins, J. K., Hulchafo, I. I., Barcelona, V., Davoudi, A., Moen, H., Harkins, S., Scharp, D., Cato, K., Tadiello, M., & Topaz, M. (2026) · JMIR Medical Informatics · doi:10.2196/85088
Yue, D., Wei, J., Qiao, R., Gao, Y., Cai, X., & Bai, T. (2026) · Cluster Computing · doi:10.1007/s10586-026-06625-5
Spectral-Semantic Consensus for Few-Shot Whole Slide Image Classification
Du, H., Zhou, P., Shi, W., Wei, Z., Chen, Y., Du, B., & Xu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38224-5_48
Madeira, C., Dopke, L., Losch, G., Magnaguagno, M. C., & Griebler, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39898-7_18
Preserving Computer Music Heritage Through RAG-Based Interpretation of Historical Source Code
Zuccante, A., & Canazza, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-40774-0_52
Artificial intelligence, the knowledge problem, and academic judgment: an Austrian perspective
Wang, W. H., & Neira, M. Á. A. (2026) · The Review of Austrian Economics · doi:10.1007/s11138-026-00747-0
Abhijeet, K., Ajay, M., Reddy, M. L. N., Santosh, D. T., & S, R. M. (2026) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci71772.2026.11710772
Embedding Collision Attacks: A Representation-Level Threat to NLP Systems
Saha, B., Rani, N., Chatterjee, U., & Shukla, S. K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38702-8_7
Göksel, G. (2026) · Web Intelligence · doi:10.1177/24056456261497567
Entropy evolution in multilingual transformer models : An information-theoretic perspective
Bansode, A., & Chaware, A. (2026) · Journal of Information and Optimization Sciences · doi:10.47974/jios-2353
What a Secret Rotation Buys: Exact Retrieval and Measured Leakage in Vector Databases
Cernica, I., & Popescu, N. (2026) · Applied Sciences · doi:10.3390/app162010259
A topic-based framework for interpretable and sparse semantic representations
Song, M., & Cai, Q. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03187-7
SWIS: An Efficient Structured Pruning Framework for LLMs with Spectral Geometric Alignment
Xu, L., & Li, P. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729617
False Proximity: Frozen Code Embeddings Invert Similarity on Behaviourally Divergent Mutant PairsPreprint
Letsholo, K. J., Ayed, N. E. B., Ghorbel, A., & Alrammal, M. (2026) · Research Square · doi:10.21203/rs.3.rs-10188373/v1
Kenyon, C., & Stine, Z. K. (2026) · Southeast Symposium on Contemporary Engineering Topics (SSCET) · doi:10.1109/sscet69323.2026.11735532
The latent structure of diversification from relatedness and network embeddings
Sturm, N., Candia, C., Damásio, B., & Pinheiro, F. L. (2026) · PLoS ONE · doi:10.1371/journal.pone.0344958
When Data Matters More Than Models: Offensive Language Detection in Low-Resource Malayalam
Nair, S. B., Thomas, N., Jose, R. T., Sojan, R. A., Rawther, F. A., & Titus, G. (2026) · IEEE International Conference on Recent Advances in Electronics, Communication, Computing, Automation and Power (RECCAP) · doi:10.1109/reccap69368.2026.11723371
Adoptante, A. J. M., Medrana, M. L., Deocadiz, A., Peramo, E. C., & Ocampo, A. L. P. D. (2026) · International Conference on Machine Learning and Machine Intelligence (MLMI) · doi:10.1145/3841358.3841368
Pascale, F. D., Cuconato, S., Bona, M., & Ferrari, D. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-29322-0_19
VenusGenome: Genome-Aware Masked Autoencoding for Microbial Genome Representation LearningPreprint
Zhang, L., Zhang, Y., Tan, Y., Jiang, F., Zhong, B., Cheng, X., Hong, L., Li, M., & Yu, Y. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.24.761516
Beyond Adapter Expansion: Packing Multiple Tasks into Spare LoRA Capacity
Fan, S., Huang, C., Zhao, Z., Koniusz, P., Gao, J., & Hu, D. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1570
Bardhan, R., Sumanth, V., Dorai, G., Regmi, P., & Kumar, P. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8110359
Rosen, Z. P., & Dale, R. A. C. (2026) · Journal of Language Evolution · doi:10.1093/jole/lzag009
Geometric Recalibration of IndoBERT Embeddings for Classification of Short Indonesian Academic Texts
Bengnga, A., Ilham, A. A., & Mokobombang, N. N. (2026) · Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS) · doi:10.1109/eeccis68982.2026.11750869
Hicham, N., & Habbat, N. (2026) · Data Science and Informetrics · doi:10.1016/j.dsim.2026.100059
Lopez, J. M. H., & Espinosa, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-44176-8_38
Animacy annotation for ancient Greek: comparing two transformer-based approaches
Keersmaekers, A. (2026) · Language Resources and Evaluation · doi:10.1007/s10579-026-09962-4
Election Sentiment Analysis: A Systematic Review of AI-Driven Public Opinion Mining
Kumari, J., Nandan, D., & Singh, M. (2026) · doi:10.1109/icact70625.2026.11748557
Incomplete Single-Vector Readiness in Dense Retrieval: Information Condensation Across Depth
Wang, R., & Li, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-7738-4_24
Diagnosing and Mitigating Entity Collapse in Short-Text Entity Matching for Reliable Data Governance
Mao, W.-H., Yin, R., Liu, Y., Zheng, J., & 杨, 政. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-7741-4_29
Metadata
- Authors
- Ethayarajh, K.
- Affiliations
- Stanford University (US)
- Corresponding author
- Ethayarajh, Kawin
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 55-65
- OpenAlex
- W2971569798
- Cited by
- 841
- References
- 27
- Citations by year
- 2026: 93 · 2025: 92 · 2024: 115 · 2023: 160 · 2022: 158 · 2021: 157 · 2020: 64
- FWCI
- 36.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- contextualized word representations, word embeddings, BERT, ELMo, GPT-2, embedding geometry, cosine similarity, static embeddings, layer-wise analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
