The DOI matches its Crossref record; 54 of the 57 references it cites were found in a registry or an index.
Dissecting Contextual Word Embeddings: Architecture and Representation
Open accessAbstract
Contextual word representations derived from pre-trained bidirectional language models (biLMs) have recently been shown to provide significant improvements to the state of the art for a wide range of NLP tasks.However, many questions remain as to how and why these models are so effective.In this paper, we present a detailed empirical study of how the choice of neural architecture (e.g.LSTM, CNN, or self attention) influences both end task accuracy and qualitative properties of the representations that are learned.We show there is a tradeoff between speed and accuracy, but all architectures learn high quality contextual representations that outperform word embeddings for four challenging NLP tasks.Additionally, all architectures learn representations that vary with network depth, from exclusively morphological based at the word embedding layer through local syntax based in the lower contextual layers to longer range semantics such coreference at the upper layers.Together, these results suggest that unsupervised biLMs, independent of architecture, are learning much more about the structure of language than previously appreciated.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Following Peters et al. (2018b); Liu et al. (2019a), we observe that the final 2-3 BERT layers provide embeddings that excel at predictive language modeling; as such, Figure 4 uses embeddings from layer 9 of S2ORC-SCIBERT.”
From S2ORC: The Semantic Scholar Open Research Corpus (2020) · cites this paper as Peters et al. (2018b)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The study by Peters et al. (2018) suggests that the context vectors of bi-directional language models undergo abrupt changes at syntactic boundaries, allowing them to be utilized for constructing representations of spans or consecutive token sequences.”
From Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis (2025) · cites this paper as Peters et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Self-citationBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Research works printed but not linked: 4
- Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Has- san Sajjad, and James R. Glass. 2017. What do neu- ral machine translation models learn about morphol- ogy? In ACL.
- Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open- NMT: Open-source toolkit for neural machine trans- lation. In ACL.
- Liyuan Liu, Xiang Ren, Jingbo Shang, Jian Peng, and Jiawei Han. 2018. Efficient contextualized represen- tation: Language model pruning for sequence label- ing. In EMNLP.
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In NAACL.
Cited by
Sorted by citations
Items 201–300 of 436
Interpretability of deep learning models in analysis of Spanish financial text
Vaca, C., Astorgano, M., Rivero, A. J. L., Gaite, F. T., & Sahelices, B. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09474-8 · Cited by 13
Jettakul, A., Wichadakul, D., & Vateekul, P. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3217-3 · Cited by 12
Semantics-aware Attention Improves Neural Machine Translation
Slobodkin, A., Choshen, L., & Abend, O. (2022) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2022.starsem-1.3 · Cited by 10
Modelling Semantic Categories Using Conceptual Neighborhood
Bouraoui, Z., Camacho-Collados, J., Espinosa-Anke, L., & Schockaert, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6241 · Cited by 10
How Many Layers and Why? An Analysis of the Model Depth in Transformers
Simoulin, A., & Crabbé, B. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-srw.23 · Cited by 15
CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learning
Suglia, A., Konstas, I., Vanzo, A., Bastianelli, E., Elliott, D., Frank, S., & Lemon, O. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.682 · Cited by 10
Shallow Syntax in Deep WaterPreprint
Swayamdipta, S., Peters, M. E., Roof, B., Dyer, C., & Smith, N. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11047 · Cited by 12
Self-citationBarrière, V., & Fouret, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03453 · Cited by 10
Choenni, R., & Shutova, E. (2022) · Computational Linguistics · doi:10.1162/coli_a_00444 · Cited by 11
Xue, J., Du, X., Zhao, L., Yang, Z., Xia, C., Ma, Y., Hoque, M. J., Fu, W., Yan, X., & Miljkovic, N. (2025) · Energy and AI · doi:10.1016/j.egyai.2025.100549 · Cited by 9
Evaluation of taxonomic and neural embedding methods for calculating semantic similarity
Yang, D., & Yin, Y. (2021) · Natural Language Engineering · doi:10.1017/s1351324921000279 · Cited by 9
How to Leverage a Multi-layered Transformer Language Model for Text Clustering
Ait-Saada, M., Role, F., & Nadif, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482121 · Cited by 11
Building a Vietnamese Dataset for Natural Language Inference Models
Nguyen, C. T., & Nguyen, D. T. (2022) · SN Computer Science · doi:10.1007/s42979-022-01267-x · Cited by 9
Hierarchical features extraction and data reorganization for code search
Zhang, F., Peng, M., Shen, Y., & Wu, Q. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111896 · Cited by 10
INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers
Karmakar, A., & Robbes, R. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3341624 · Cited by 9
Comparing general and specialized word embeddings for biomedical named entity recognition
Ramos-Vargas, R., Román-Godínez, I., & Torres-Ramos, S. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.384 · Cited by 9
Kementchedjhieva, Y., Anderson, M., & Søgaard, A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.429 · Cited by 10
Graph Representation Learning in Document Wikification
Saeidi, M., Milios, E., & Zeh, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86159-9_37 · Cited by 9
Probing Pre-trained Language Models for Semantic Attributes and their Values
Beloucif, M., & Biemann, C. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.218 · Cited by 10
Deep Clustering of Text Representations for Supervision-Free Probing of Syntax
Gupta, V., Shi, H., Gimpel, K., & Sachan, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21317 · Cited by 11
PinText: A Multitask Text Embedding System in Pinterest
Zhuang, J., & Liu, Y. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330671 · Cited by 9
Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification
Moore, A., & Barnes, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.227 · Cited by 10
Comparative Analysis of Fine-Tuned LLM, BERT and DL Models for Customer Sentiment Analysis
Chinnalagu, A. (2024) · International Conference on System Modeling & Advancement in Research Trends (SMART) · doi:10.1109/smart63812.2024.10882546 · Cited by 9
Petilli, M. A., Marelli, M., Mazzoni, G. A. L., Marchetti, M., Rinaldi, L., & Gatti, D. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02425-0 · Cited by 9
CL-IMS @ DIACR-Ita: Volente o Nolente: BERT does not Outperform SGNS on Semantic Change Detection
Laicher, S., Baldissin, G., Castañeda, E., Schlechtweg, D., & Walde, S. S. I. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.7650 · Cited by 8
Efficient Contextual Representation Learning With Continuous Outputs
Li, L. H., Chen, P. H., Hsieh, C., & Chang, K.-W. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00289 · Cited by 10
Kryeziu, L., & Shehu, V. (2022) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco55406.2022.9797081 · Cited by 9
BERT as a Teacher: Contextual Embeddings for Sequence-Level RewardPreprint
Schmidt, F., & Hofmann, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02738 · Cited by 9
BERTnesia: Investigating the capture and forgetting of knowledge in BERTPreprint
Wallat, J., Singh, J., & Anand, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09313 · Cited by 18
AI Translation Tools in Language Learning
Ayyaz, S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9077-1.ch003 · Cited by 7
Context, not grammar, is key to structural priming
Nielsen, Y. A., & Christiansen, M. H. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.05.016 · Cited by 7
Emergent Structures and Training Dynamics in Large Language Models
Teehan, R., Clinciu, M., Serikov, O., Szczechla, E., Seelam, N., Mirkin, S., & Gokaslan, A. (2022) · doi:10.18653/v1/2022.bigscience-1.11 · Cited by 9
An Exploration of Prompt-Based Zero-Shot Relation Extraction Method
Zhao, J., Hu, Y., Xu, N., Gui, T., Zhang, Q., Chen, Y., & Gao, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18315-7_6 · Cited by 7
Atutxa, A., Bengoetxea, K., de Ilarraza, A. D., & Iruskieta, M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221639 · Cited by 7
How Pre-trained Word Representations Capture Commonsense Physical Comparisons
Goel, P., Feng, S., & Boyd-Graber, J. L. (2019) · doi:10.18653/v1/d19-6016 · Cited by 7
Synthetic Propaganda Embeddings To Train A Linear Projection
Ek, A., & Ghanimifard, M. (2019) · Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda (NLP4IF) · doi:10.18653/v1/d19-5023 · Cited by 7
Ranking professional forecasters by the predictive power of their narratives
Rybinski, K. I. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2020.04.003 · Cited by 7
Pre-trained Language Models with Limited Data for Intent Classification
Kasthuriarachchy, B. H., Chetty, M., Karmakar, G., & Walls, D. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207121 · Cited by 8
Robust Identification of Figurative Language in Personal Health Mentions on Twitter
Naseem, U., Kim, J., Khushi, M., & Dunn, A. G. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3175469 · Cited by 7
An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale
Liu, Y., Dmitriev, P., Huang, Y., Brooks, A., & Li, D. (2019) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2019.00069 · Cited by 7
Inspecting the concept knowledge graph encoded by modern language models
Aspillaga, C., Mendoza, M., & Soto, Á. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.263 · Cited by 9
Sex and gender bias in natural language processing
Cirillo, D., Gonen, H., Santus, E., Valencia, A., Costa‐jussà, M. R., & Villegas, M. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-821392-6.00009-1 · Cited by 8
A Picture May Be Worth a Hundred Words for Visual Question Answering
Hirota, Y., García, N., Otani, M., Chu, C., & Nakashima, Y. (2024) · Electronics · doi:10.3390/electronics13214290 · Cited by 8
Deepening Hidden Representations from Pre-trained Language ModelsPreprint
Yang, J., & Zhao, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01940 · Cited by 7
Berenguer, A., Alcaraz, O., Tomás, D., & Mazón, J.-N. (2024) · IEEE Software · doi:10.1109/ms.2024.3359333 · Cited by 6
Transformer-based Language models for the Identification of Idiomatic Expressions
Arachchige, I. A. N., Suraweera, S. M. D. A. S., & Herath, D. L. (2022) · International Conference EUROPHRAS · doi:10.26615/978-954-452-080-9_015 · Cited by 6
De-identification techniques to preserve privacy in medical records
Catelli, R., & Esposito, M. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-90531-2.00007-2 · Cited by 6
Automated labeling of PDF mathematical exercises with word N-grams VSM classification
Yamauchi, T., Flanagan, B., Nakamoto, R., Dai, Y., Takami, K., & Ogata, H. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00271-9 · Cited by 6
M2C: Concise Music Representation for 3D Dance Generation
Marchellus, M., & Park, I. K. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00337 · Cited by 6
Improving Biomedical Analogical Retrieval with Embedding of Structural Dependencies
Paullada, A., Percha, B., & Cohen, T. (2020) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2020.bionlp-1.4 · Cited by 6
On Incorporating Structural Information to improve Dialogue Response Generation
Moghe, N., Vijayan, P., Ravindran, B., & Khapra, M. M. (2020) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2020.nlp4convai-1.2 · Cited by 6
From General Language Understanding to Noisy Text Comprehension
Kasthuriarachchy, B. H., Chetty, M., Shatte, A. B. R., & Walls, D. (2021) · Applied Sciences · doi:10.3390/app11177814 · Cited by 6
Unsupervised Transfer Learning via BERT Neuron SelectionPreprint
Valipour, M., Lee, E.-S. A., Jamacaro, J. R., & Bessega, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.05308 · Cited by 7
Environmental due diligence data: A novel corpus for training environmental domain NLP models
Aman, A., & Reji, D. J. (2022) · Data in Brief · doi:10.1016/j.dib.2022.108579 · Cited by 6
Retrieval-Augmented Knowledge-Intensive Dialogue
Wang, Z., Gong, P., Zhang, Y., Gu, J., & Yang, X. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44693-1_2 · Cited by 6
Xiao, X., Li, B., Wu, C., Tseng, S.-Y., Bhiwandiwalla, A., Rosenman, S., Lal, V., Che, W., & Duan, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.811 · Cited by 5
Mitigating Hallucination Issues in Small-Parameter LLMs through Inter-Layer Contrastive Decoding
Li, F., zhang, X., & Zhang, P. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650644 · Cited by 5
Casting Light on Invisible Cities: Computationally Engaging with Literary Criticism
Wang, S., & Iyyer, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1130 · Cited by 5
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Zhao, M., Dufter, P., Yaghoobzadeh, Y., & Schütze, H. (2020) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.74039 · Cited by 5
Comparing Knowledge-Intensive and Data-Intensive Models for English Resource Semantic Parsing
Cao, J., Lin, Z. J., Sun, W., & Wan, X. (2021) · Computational Linguistics · doi:10.1162/coli_a_00395 · Cited by 5
Pavlova, V., & Makhlouf, M. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.31 · Cited by 5
A Text Classification Model Based on BERT and Attention
Zhu, B., & Wei, P. (2023) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait59945.2023.10469363 · Cited by 5
DESCGEN: A Distantly Supervised Datasetfor Generating Entity Descriptions
Shi, W., Joshi, M., & Zettlemoyer, L. (2021) · doi:10.18653/v1/2021.acl-long.35 · Cited by 5
Self-citationVarying Abstractions: a conceptual vs. distributional view on prepositional polysemy
Fonteyn, L. (2021) · Glossa a journal of general linguistics · doi:10.5334/gjgl.1323 · Cited by 9
Sentiment Analysis on Mixed-Languages Customer Reviews: A Hybrid Deep Learning Approach
Durairaj, A. K., & Chinnalagu, A. (2021) · International Conference on System Modeling & Advancement in Research Trends (SMART) · doi:10.1109/smart52563.2021.9676277 · Cited by 5
Fast PMI-Based Word Embedding with Efficient Use of Unobserved Patterns
Soleimani, B. H., & Matwin, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017031 · Cited by 5
Using Multi-Sense Vector Embeddings for Reverse DictionariesPreprint
Hedderich, M. A., Yates, A., Klakow, D., & de Melo, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01451 · Cited by 7
When is it permissible for artificial intelligence to lie? A trust-based approachPreprint
Kim, T. W., Lu, T., Lu, Lee, K., Cheng, Z., Tang, Y., & Hooker, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05434 · Cited by 6
Using Machine Learning to Extract Insights From Consumer Data
Chang, H. H., & Mukherjee, A. (2023) · IGI Global eBooks · doi:10.4018/978-1-7998-9220-5.ch107 · Cited by 5
Zhao, Y. D., Gong, L., Yu, Y., Huang, Z., & Wei, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126762 · Cited by 4
EnvBert: An NLP model for Environmental Due Diligence data classification
Aman, A., & Reji, D. J. (2022) · Software Impacts · doi:10.1016/j.simpa.2022.100427 · Cited by 4
Aishwarya, G., Kavitha, B., Mittalakod, P. S., Sunitha, K., & Ambore, B. (2023) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i59117.2023.10397916 · Cited by 4
RetractedThe Graph Reasoning Approach Based on the Dynamic Knowledge Auxiliary for Complex Fact Verification
Wang, Y., Xia, C., Si, C., Zhang, C., & Wang, T. (2020) · Electronics · doi:10.3390/electronics9091472 · Cited by 4
Zhang, Z., Fang, M., Wu, R. C., Zong, H., Huang, H., Tong, Y., Xie, Y., Cheng, S., Wei, Z., Crabbe, M. J. C., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/48115 · Cited by 5
Emergence of Syntax Needs Minimal Supervision
Bailly, R., & Gábor, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.46 · Cited by 4
Encodings of Source Syntax: Similarities in NMT Representations Across Target Languages
Chang, T. A., & Rafferty, A. N. (2020) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2020.repl4nlp-1.2 · Cited by 4
A Vietnamese Answer Extraction Model Based on PhoBERT
Nguyen, C. T., & Nguyen, D. T. (2021) · International Conference on Advanced Computing and Applications (ACOMP) · doi:10.1109/acomp53746.2021.00022 · Cited by 4
From BERT‘s Point of View: Revealing the Prevailing Contextual Differences
Schuster, C. M., & Hegelich, S. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.89 · Cited by 5
Biomedical Word Sense Disambiguation with Contextualized Representation Learning
Saeidi, M., Milios, E., & Zeh, N. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524703 · Cited by 4
Contextual word embeddings for tabular data search and integration
Pilaluisa, J., Tomás, D., Navarro-Colorado, B., & Mazón, J.-N. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-08066-8 · Cited by 4
Effective Sentence Scoring Method using Bidirectional Language Model for Speech RecognitionPreprint
Shin, J., Lee, Y., & Jung, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06655 · Cited by 4
What do Deep Networks Like to Read?Preprint
Pfeiffer, J., Kamath, A., Gurevych, I., & Ruder, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04547 · Cited by 4
Specializing Word Embeddings (for Parsing) by Information BottleneckPreprint
Li, X. L., & Eisner, J. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00163 · Cited by 4
Geiger, A., Richardson, K., & Potts, C. (2020) · arXiv (Cornell University) · Cited by 4
An exploration of the encoding of grammatical gender in word embeddingsPreprint
Veeman, H., & Basirat, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01946 · Cited by 4
Li, J. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100163 · Cited by 3
Gupta, R., Tiwari, S. M., & Chaudhary, P. (2025) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-82062-5_5 · Cited by 3
Farhan, M., Ahmed, A., Chong, K. T., & Tayara, H. (2025) · Asia-Pacific Financial Markets · doi:10.1007/s10690-025-09560-4 · Cited by 4
Fake News Detection Using LSTM-Based Deep Learning Approach and Word Embedding Feature Extraction
Bankar, S. M., & Gupta, S. K. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-1699-3_8 · Cited by 3
A hybrid model to improve IC-related metrics of semantic similarity between words
Xiao, J. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01496-y · Cited by 3
From words to senses: A sense-based approach to quantitative polysemy detection across disciplines
Geng, W., & Liang, M. (2024) · Journal of English for Academic Purposes · doi:10.1016/j.jeap.2024.101449 · Cited by 4
A Survey of Coreference and Zeros Resolution for Arabic
Aloraini, A., Yu, J., Aliady, W. A., & Poesio, M. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3702323 · Cited by 3
Classifying Genetic Essentialist Biases using Large Language Models
Reimann, R., Lynch, K. E., Gawronski, S. A., Chan, J., & Griffiths, P. E. (2025) · Review of Philosophy and Psychology · doi:10.1007/s13164-025-00770-3 · Cited by 3
Emergence of a temporal processing gradient from naturalistic inputs and network connectivity
Chang, C. H. C., Nastase, S. A., Hasson, U., & Dominey, P. F. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420105122 · Cited by 5
Dong, B., Su, H., Xu, D., Hou, C., Liu, Z., Niu, N., & Wang, G. (2024) · Genes · doi:10.3390/genes15101350 · Cited by 3
Analysing Word Representation from the Input and Output Embeddings in Neural Network Language Models
Derby, S., Miller, P. C., & Devereux, B. J. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.36 · Cited by 3
Unsupervised Methods for the Study of Transformer Embeddings
Ait-Saada, M., Role, F., & Nadif, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-74251-5_23 · Cited by 3
Contextualized Knowledge Base Sense Embeddings in Word Sense Disambiguation
Saeidi, M., Milios, E., & Zeh, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86159-9_12 · Cited by 3
Metadata
- Authors
- Peters, M., Neumann, M., Zettlemoyer, L., & Yih, W.-T.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 1499-1509
- OpenAlex
- W2888329843
- Cited by
- 434
- References
- 57
- Citations by year
- 2026: 23 · 2025: 27 · 2024: 37 · 2023: 53 · 2022: 55 · 2021: 75 · 2020: 83 · 2019: 77 · 2018: 4
- FWCI
- 24.08
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- contextual word embeddings, bidirectional language models, LSTM, convolutional neural networks, self-attention, representation learning, network depth, morphological representation, syntactic representation, coreference, word embeddings, speed-accuracy trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
