Designing and Interpreting Probes with Control Tasks

Adam: A Method for Stochastic Optimization

Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Glove: Global Vectors for Word Representation

Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works

Untitled work

(2014) · Cited by 33,419 · see citing works

Distributed Representations of Words and Phrases and their Compositionality

Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works

Deep Contextualized Word Representations

Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works

Building a Large Annotated Corpus of English: The Penn Treebank

Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405 · see citing works

Understanding deep learning requires rethinking generalization

Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · see citing works

Generating Typed Dependency Parses from Phrase Structure Parses

de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3wtkaidk6ea8 · Cited by 2,289 · see citing works

Rademacher and Gaussian Complexities: Risk Bounds and Structural Results

Bartlett, P. L., & Mendelson, S. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44581-1_15 · Cited by 1,989 · see citing works

Linguistic Knowledge and Transferability of Contextual Representations

Liu, N. F., Gardner, M., Belinkov, Y., Peters, M. E., & Smith, N. A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1112 · Cited by 651 · see citing works

Untitled work

(2019) · doi:10.18653/v1/n19-1419 · Cited by 601 · see citing works

Understanding intermediate layers using linear classifier probes

Alain, G., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1610.01644 · Cited by 581 · see citing works

What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties

Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1198 · Cited by 572 · see citing works

Analysis Methods in Neural Language Processing: A Survey

Belinkov, Y., & Glass, J. R. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00254 · Cited by 485 · see citing works

What do you learn from context? Probing for sentence structure in contextualized word representationsPreprint

Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Durme, B. V., Bowman, S. R., Das, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06316 · Cited by 480 · see citing works

Dissecting Contextual Word Embeddings: Architecture and Representation

Peters, M., Neumann, M., Zettlemoyer, L., & Yih, W.-T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1179 · Cited by 434 · see citing works

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint

Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04207 · Cited by 374 · see citing works

What you can cram into a single vector: Probing sentence embeddings for linguistic propertiesPreprint

Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01070 · Cited by 315 · see citing works

What do Neural Machine Translation Models Learn about Morphology?

Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., & Glass, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1080 · Cited by 307 · see citing works

Does String-Based Neural MT Learn Source Syntax?

Shi, X., Padhi, I., & Knight, K. K. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1159 · Cited by 302 · see citing works

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint

Adi, Y. (2024) · arXiv (Cornell University) · doi:10.57702/tcr06vve · Cited by 228 · see citing works

Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information

Giulianelli, M., Harding, J., Mohnert, F., et al. (2018) · BlackboxNLP@EMNLP · doi:10.18653/v1/w18-5426 · Cited by 207

Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging TasksPreprint

Belinkov, Y., Màrquez, L., Sajjad, H., Durrani, N., Dalvi, F., & Glass, J. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.07772 · Cited by 146 · see citing works

Probing for semantic evidence of composition by means of simple classification tasks

Ettinger, A., Elgohary, A., & Resnik, P. (2016) · Workshop on Evaluating Vector-Space Representations for NLP · doi:10.18653/v1/w16-2524 · Cited by 131 · see citing works

Probing What Different NLP Tasks Teach Machines about Function Word Comprehension

Kim, N., Patel, R., Poliak, A., Xia, P., Wang, A., McCoy, T., Tenney, I., Ross, A., Linzen, T., Durme, B. V., et al. (2019) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s19-1026 · Cited by 89 · see citing works

Distributional vectors encode referential attributes

Gupta, A., Boleda, G., Baroni, M., & Padó, S. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1002 · Cited by 70 · see citing works

Analyzing Hidden Representations in End-to-End Automatic Speech Recognition SystemsPreprint

Belinkov, Y., & Glass, J. R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.04482 · Cited by 66 · see citing works

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task AnalysisPreprint

Zhang, K. W., & Bowman, S. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10040 · Cited by 45 · see citing works

Investigating Language Universal and Specific Properties in Word Embeddings

Qian, P., Qiu, X., & Huang, X. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1140 · Cited by 42 · see citing works

Assessing Composition in Sentence Vector RepresentationsPreprint

Ettinger, A., Elgohary, A., Phillips, C., & Resnik, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03992 · Cited by 35 · see citing works

LINSPECTOR: Multilingual Probing Tasks \nfor Word Representations

Şahin, G. G., Vania, C., Kuznetsov, I., & Gurevych, I. (2019) · TUbilio (Technical University of Darmstadt) · doi:10.1162/coli_a_00376 · Cited by 32 · see citing works

Understanding Learning Dynamics Of Language Models with SVCCA

Saphra, N., & Lopez, A. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1329 · Cited by 31 · see citing works

The Lazy Encoder: A Fine-Grained Analysis of the Role of Morphology in Neural Machine Translation

Bisazza, A., & Tump, C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1313 · Cited by 24 · see citing works

Arc-swift: A Novel Transition System for Dependency Parsing

Qi, P., & Manning, C. D. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-2018 · Cited by 23 · see citing works

Analyzing Learned Representations of a Deep ASR Performance Prediction Model

Elloumi, Z., Besacier, L., Galibert, O., & Lecouteux, B. (2018) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w18-5402 · Cited by 21 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805

On the Opportunities and Risks of Foundation Models

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279 · see citing works

Explainability for Large Language Models: A Survey

Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681 · see citing works

FNet: Mixing Tokens with Fourier Transforms

Lee-Thorp, J., Ainslie, J., Eckstein, I., & Ontañón, S. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.319 · Cited by 575 · see citing works

Factual Probing Is [MASK]: Learning vs. Learning to Recall

Zhong, Z., Friedman, D., & Chen, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.398 · Cited by 274 · see citing works

oLMpics-On What Language Model Pre-training Captures

Talmor, A., Elazar, Y., Goldberg, Y., & Berant, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00342 · Cited by 267 · see citing works

Probing Classifiers: Promises, Shortcomings, and Advances

Belinkov, Y. (2021) · Computational Linguistics · doi:10.1162/coli_a_00422 · Cited by 311 · see citing works

Revisiting Few-sample BERT Fine-tuningPreprint

Zhang, T., Wu, F., Katiyar, A., Weinberger, K. Q., & Artzi, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05987 · Cited by 295 · see citing works

From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers

Lauscher, A., Ravishankar, V., Vulić, I., & Glavaš, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.363 · Cited by 250 · see citing works

Hydrological concept formation inside long short-term memory (LSTM) networks

Lees, T., Reece, S., Kratzert, F., Klotz, D., Gauch, M., de Bruijn, J., Sahu, R. K., Greve, P., Slater, L. J., & Dadson, S. (2022) · Hydrology and earth system sciences · doi:10.5194/hess-26-3079-2022 · Cited by 254 · see citing works

Post-hoc Interpretability for Neural NLP: A Survey

Madsen, A., Reddy, S., & Chandar, S. (2022) · ACM Computing Surveys · doi:10.1145/3546577 · Cited by 229 · see citing works

Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa

Dai, J., Yan, H. R., Sun, T., Liu, P., & Qiu, X. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.146 · Cited by 190 · see citing works

Information-Theoretic Probing with Minimum Description Length

Voita, E., & Titov, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.14 · Cited by 197 · see citing works

Information-Theoretic Probing for Linguistic Structure

Pimentel, T., Valvoda, J., Maudslay, R. H., Zmigrod, R., Williams, A., & Cotterell, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.420 · Cited by 173 · see citing works

Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT

Wu, Z., Chen, Y., Kao, B., & Liu, Q. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.383 · Cited by 159 · see citing works

Interpretability research of deep learning: A literature survey

Xu, B., & Yang, G. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102721 · Cited by 159 · see citing works

What Happens To BERT Embeddings During Fine-tuning?

Merchant, A., Rahimtoroghi, E., Pavlick, E., & Tenney, I. F. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.4 · Cited by 159 · see citing works

Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings

Bommasani, R., Davis, K., & Cardie, C. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.431 · Cited by 147 · see citing works

Untitled work

(2020) · neural information processing systems · Cited by 142 · see citing works

Towards Faithful Model Explanation in NLP: A Survey

Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2024) · Computational Linguistics · doi:10.1162/coli_a_00511 · Cited by 143 · see citing works

Finding Universal Grammatical Relations in Multilingual BERT

Chi, E. A., Hewitt, J., & Manning, C. D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.493 · Cited by 127 · see citing works

The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders

He, H. W., & Choi, J. D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.451 · Cited by 130 · see citing works

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals

Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00359 · Cited by 126 · see citing works

A Survey on Contextual EmbeddingsPreprint

Liu, Q., Kusner, M. J., & Blunsom, P. (2020) · arXiv (Cornell University) · doi:10.4230/oasics.slate.2021.12 · Cited by 178 · see citing works

What Artificial Neural Networks Can Tell Us about Human Language Acquisition

Warstadt, A., & Bowman, S. R. (2022) · Algebraic Structures in Natural Language · doi:10.1201/9781003205388-2 · Cited by 111 · see citing works

GenePT: A Simple But Effective Foundation Model for Genes and Cells Built From ChatGPTPreprint

Chen, Y. T., & Zou, J. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.16.562533 · Cited by 106 · see citing works

Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions

Han, X., Wallace, B., & Tsvetkov, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.492 · Cited by 111 · see citing works

Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)

Warstadt, A., Zhang, Y., Li, H.-S., Liu, H., & Bowman, S. R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.16 · Cited by 106 · see citing works

Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections

Zhong, R., Lee, K., Zhang, Z., & Klein, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.244 · Cited by 118 · see citing works

Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers

Hendricks, L. A., Mellor, J., Schneider, R., Alayrac, J.-B., & Nematzadeh, A. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00385 · Cited by 100 · see citing works

BERT4GCN: Using BERT Intermediate Layers to Augment GCN for Aspect-based Sentiment Classification

Xiao, Z., Wu, J., Chen, Q., & Deng, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.724 · Cited by 96 · see citing works

When Do You Need Billions of Words of Pretraining Data?

Zhang, Y., Warstadt, A., Li, X., & Bowman, S. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.90 · Cited by 85 · see citing works

Improving LSTM hydrological modeling with spatiotemporal deep learning and multi-task learning: A case study of three mountainous areas on the Tibetan Plateau

Li, B., Li, R., Sun, T., Gong, A., Tian, F., Khan, M. Y. A., & Ni, G. (2023) · Journal of Hydrology · doi:10.1016/j.jhydrol.2023.129401 · Cited by 88 · see citing works

Contrastive Explanations for Model Interpretability

Jacovi, A., Swayamdipta, S., Ravfogel, S., Elazar, Y., Choi, Y., & Goldberg, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.120 · Cited by 83 · see citing works

Implicit Representations of Meaning in Neural Language Models

Li, B. Z., Nye, M., & Andreas, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.143 · Cited by 84 · see citing works

Iteratively Prompt Pre-trained Language Models for Chain of Thought

Wang, B., Deng, X. B., & Sun, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.174 · Cited by 89 · see citing works

Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training

Shi, P., Ng, P., Wang, Z., Zhu, H., Li, A. H., Wang, J., dos Santos, C. N., & Xiang, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17627 · Cited by 73 · see citing works

Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation

Miaschi, A., & Dell’Orletta⋄, F. (2020) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2020.repl4nlp-1.15 · Cited by 75 · see citing works

Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?

Ravichander, A., Belinkov, Y., & Hovy, E. H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.295 · Cited by 75 · see citing works

Semantic Structure in Deep Learning

Pavlick, E. (2021) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-031120-122924 · Cited by 76 · see citing works

Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color

Abdou, M., Kulmizev, A., Hershcovich, D., Frank, S., Pavlick, E., & Søgaard, A. (2021) · Proceedings of the 25th Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.9 · Cited by 64 · see citing works

Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender BiasPreprint

Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang, J., Singer, Y., & Shieber, S. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12265 · Cited by 63 · see citing works

Analyzing Redundancy in Pretrained Transformer Models

Dalvi, F., Sajjad, H., Durrani, N., & Belinkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.398 · Cited by 71 · see citing works

Language Model Evaluation Beyond Perplexity

Meister, C., & Cotterell, R. (2021) · doi:10.18653/v1/2021.acl-long.414 · Cited by 74 · see citing works

BERT syntactic transfer: A computational experiment on Italian, French and English languages

Guarasci, R., Silvestri, S., Pietro, G. D., Fujita, H., & Esposito, M. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101261 · Cited by 55 · see citing works

Analyzing Individual Neurons in Pre-trained Language Models

Durrani, N., Sajjad, H., Dalvi, F., & Belinkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.395 · Cited by 60 · see citing works

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks

Wu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J., & Kim, Y. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.102 · Cited by 58 · see citing works

Untitled work

(2021) · doi:10.18653/v1/2021.conll-1.15 · Cited by 51 · see citing works

Local Interpretations for Explainable Natural Language Processing: A Survey

Luo, S., Ivison, H., Han, S. C., & Poon, J. (2024) · ACM Computing Surveys · doi:10.1145/3649450 · Cited by 60 · see citing works

First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT

Müller, B., Elazar, Y., Sagot, B., & Seddah, D. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.189 · Cited by 55 · see citing works

A Tale of a Probe and a Parser

Maudslay, R. H., Valvoda, J., Pimentel, T., Williams, A., & Cotterell, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.659 · Cited by 49 · see citing works

Proceedings of the 5th Workshop on Representation Learning for NLP

(2020) · doi:10.18653/v1/2020.repl4nlp-1 · Cited by 50 · see citing works

Which *BERT? A Survey Organizing Contextualized Encoders

Xia, P., Wu, S., & Durme, B. V. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.608 · Cited by 50 · see citing works

Ecco: An Open Source Library for the Explainability of Transformer Language Models

Alammar, J. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.30 · Cited by 46 · see citing works

Causal abstractions of neural networks

Geiger, A., Lu, H., Icard, T., & Potts, C. (2021) · arXiv · doi:10.48550/arxiv.2106.02997 · Cited by 45 · see citing works

Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?

Merrill, W., Goldberg, Y., Schwartz, R., & Smith, N. A. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00412 · Cited by 44 · see citing works

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Finlayson, M., Mueller, A., Gehrmann, S., Shieber, S. M., Linzen, T., & Belinkov, Y. (2021) · doi:10.18653/v1/2021.acl-long.144 · Cited by 52 · see citing works

A Closer Look at How Fine-tuning Changes BERT

Zhou, Y., & Srikumar, V. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.75 · Cited by 50 · see citing works

Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation

Geiger, A., Richardson, K., & Potts, C. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.16 · Cited by 52 · see citing works

Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models

Papadimitriou, I., & Jurafsky, D. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.554 · Cited by 43 · see citing works

Pareto Probing: Trading Off Accuracy for Complexity

Pimentel, T., Saphra, N., Williams, A., & Cotterell, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.254 · Cited by 42 · see citing works

On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT

Ravichander, A., Hovy, E. H., Suleman, K., Trischler, A., & Cheung, J. C. K. (2020) · Joint Conference on Lexical and Computational Semantics · Cited by 41 · see citing works

Let’s Play Mono - Poly : BERT Can Reveal Words’ Polysemy Level and Partitionability into Senses

Soler, A. G., & Apidianaki, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00400 · Cited by 58 · see citing works

Computational Language Modeling and the Promise of In Silico Experimentation

Jain, S., Vo, V. A., Wehbe, L., & Huth, A. G. (2023) · Neurobiology of Language · doi:10.1162/nol_a_00101 · Cited by 37 · see citing works

Linguistic Profiling of a Neural Language Model

Miaschi, A., Brunato⋄, D., Dell’Orletta⋄, F., & Venturi⋄, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.65 · Cited by 42 · see citing works

Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models

Balkır, E., Kiritchenko, S., Nejadgholi, I., & Fraser, K. (2022) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2022.trustnlp-1.8 · Cited by 43 · see citing works

Human-Model Divergence in the Handling of Vagueness

Stengel-Eskin, E., Guallar-Blasco, J., & Durme, B. V. (2021) · Workshop on Understanding Implicit and Underspecified Language (UnImplicit) · doi:10.18653/v1/2021.unimplicit-1.6 · Cited by 39 · see citing works

Neuron-level Interpretation of Deep NLP Models: A Survey

Sajjad, H., Durrani, N., & Dalvi, F. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00519 · Cited by 41 · see citing works

Intrinsic Probing through Dimension Selection

Hennigen, L. T., Williams, A., & Cotterell, R. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000462314 · Cited by 38 · see citing works

Probing Across Time: What Does RoBERTa Know and When?

Liu, Z., Wang, Y., Kasai, J., Hajishirzi, H., & Smith, N. A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.71 · Cited by 42 · see citing works

What Do Self-Supervised Speech Models Know About Words?

Pasad, A., Chien, C.-M., Settle, S., & Livescu, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00656 · Cited by 36 · see citing works

Do Language Embeddings capture Scales?

Zhang, X., Ramachandran, D., Tenney, I., Elazar, Y., & Roth, D. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.439 · Cited by 37 · see citing works

What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge

Richardson, K., & Sabharwal, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00331 · Cited by 35 · see citing works

Do Neural Language Models Show Preferences for Syntactic Formalisms?

Kulmizev, A., Ravishankar, V., Abdou, M., & Nivre, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.375 · Cited by 34 · see citing works

Syntactic Structure Distillation Pretraining for Bidirectional Encoders

Kuncoro, A., Kong, L., Fried, D., Yogatama, D., Rimell, L., Dyer, C., & Blunsom, P. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00345 · Cited by 35 · see citing works

How does BERT’s attention change when you fine-tune? An analysis methodology and a case study in negation scope

Zhao, Y., & Bethard, S. J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.429 · Cited by 34 · see citing works

Dodrio: Exploring Transformer Models with Interactive Visualization

Wang, Z. J., Turko, R., & Chau, D. H. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.16 · Cited by 37 · see citing works

Assessing BERT’s ability to learn Italian syntax: a study on null-subject and agreement phenomena

Guarasci, R., Silvestri, S., Pietro, G. D., Fujita, H., & Esposito, M. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03297-4 · Cited by 33 · see citing works

Parsing as Pretraining

Vilares, D., Strzyz, M., Søgaard, A., & Gómez‐Rodríguez, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6446 · Cited by 33 · see citing works

Asking without Telling: Exploring Latent Ontologies in Contextual Representations

Michael, J., Botha, J. A., & Tenney, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.552 · Cited by 32 · see citing works

Pretrained Language Model Embryology: The Birth of ALBERT

Chiang, C.-H., Huang, S.-F., & Lee, H.-Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.553 · Cited by 34 · see citing works

Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing

Wang, L., Qin, B., Hui, B., Li, B., Yang, M., Wang, B., Li, B., Sun, J., Huang, F., Si, L., et al. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539305 · Cited by 31 · see citing works

Probing for Referential Information in Language Models

Sorodoc, I.-T., Gulordava, K., & Boleda, G. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.384 · Cited by 29 · see citing works

Word Order Does Matter and Shuffled Language Models Know It

Abdou, M., Ravishankar, V., Kulmizev, A., & Søgaard, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.476 · Cited by 29 · see citing works

Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT

Papadimitriou, I., Chi, E. A., Futrell, R., & Mahowald, K. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.215 · Cited by 30 · see citing works

Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing

Liu, Q., Yang, D., Zhang, J., Guo, J., Zhou, B., & Lou, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.100 · Cited by 29 · see citing works

Probing for the Usage of Grammatical Number

Lasri, K., Pimentel, T., Lenci, A., Poibeau, T., & Cotterell, R. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.603 · Cited by 28 · see citing works

Putting representations to use

Cao, R. (2022) · Synthese · doi:10.1007/s11229-022-03522-3 · Cited by 28 · see citing works

Eliciting Latent Predictions from Transformers with the Tuned Lens

Belrose, N., Ostrovsky, I., McKinney, L., Furman, Z., Smith, L., Halawi, D., Biderman, S., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2303.08112 · Cited by 27 · see citing works

Interpreting Deep Learning Models in Natural Language Processing: A ReviewPreprint

Sun, X., Yang, D., Li, X., Zhang, T., Meng, Y., Qiu, H., Wang, G., Hovy, E. H., & Li, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10470 · Cited by 40 · see citing works

How much pretraining data do language models need to learn syntax?

Pérez-Mayos, L., Ballesteros, M., & Wanner, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.118 · Cited by 26 · see citing works

Conditional probing: measuring usable information beyond a baseline

Hewitt, J., Ethayarajh, K., Liang, P., & Manning, C. D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.122 · Cited by 28 · see citing works

Automatic Text Evaluation through the Lens of Wasserstein Barycenters

Colombo, P., Staerman, G., Clavel, C., & Piantanida, P. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.817 · Cited by 26 · see citing works

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

Zhong, R., Ghosh, D., Klein, D., & Steinhardt, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.334 · Cited by 23 · see citing works

Do Language Embeddings capture Scales?

Zhang, X., Ramachandran, D., Tenney, I., Elazar, Y., & Roth, D. (2020) · Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/2020.blackboxnlp-1.27 · Cited by 22 · see citing works

When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM PredictionsPreprint

Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2020) · arXiv (Cornell University) · Cited by 22 · see citing works

A matter of framing: The impact of linguistic formalism on probing results

Kuznetsov, I., & Gurevych, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.13 · Cited by 20 · see citing works

Emergence of Separable Manifolds in Deep Language RepresentationsPreprint

Mamou, J., Le, H., Rio, M. D., Stephenson, C., Tang, H., Kim, Y., & Chung, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01095 · Cited by 23 · see citing works

Interpretable Entity Representations through Large-Scale Typing

Onoe, Y., & Durrett, G. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.54 · Cited by 18 · see citing works

DirectProbe: Studying Representations without Classifiers

Zhou, Y., & Srikumar, V. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.401 · Cited by 21 · see citing works

Improving Syntactic Probing Correctness and Robustness with Control Tasks

Ma, W., Wang, B. Z., Zhang, H., Wang, L., Coto‐Solano, R., Hassanpour, S., & Vosoughi, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.35

Embodied Knowledge in Natural Language Processing: A Prolog-Based Approach to Selected bAbI Tasks

Cuartero-Sánchez, J. M. (2025) · doi:10.13140/rg.2.2.29837.65763

A study on Inference Control in Natural Language Processing(自然言語処理における推論の制御に関する研究)

宮田, 高. (1996) · doi:10.11501/3139771

The integration of language and content: action research on a theory of task design

Jones, A., & Sin, S. (2007) · Journal of Applied Linguistics · doi:10.1558/japl.v1i1.95 · Cited by 2 · see citing works

Natural Language Interface Put in Perspective: Interaction of Search Method and Task Complexity

Wang, Q., Hu, J., & Nass, C. (2005) · Natural Language Processing and Cognitive Science · doi:10.5220/0002566000030012