BERTology Meets Biology: Interpreting Attention in Protein Language Models
Open accessAbstract
Transformer architectures have proven to learn useful representations for protein classification and generation tasks. However, these representations present challenges in interpretability. Through the lens of attention, we analyze the inner workings of the Transformer and explore how the model discerns structural and functional properties of proteins. We show that attention (1) captures the folding structure of proteins, connecting amino acids that are far apart in the underlying sequence, but spatially close in the three-dimensional structure, (2) targets binding sites, a key functional component of proteins, and (3) focuses on progressively more complex biophysical properties with increasing layer depth. We also present a three-dimensional visualization of the interaction between attention and protein structure. Our findings align with known biological processes and provide a tool to aid discovery in protein engineering and synthetic biology. The code for visualization and analysis is available at https://github.com/salesforce/provis .
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ProGen's model likelihoods align better with experimental assay data for CM and MDH than the original family-specific generation methods.
“In sum, ProGen’s model likelihoods are better aligned with experimentally-measured assay data on two diverse protein datasets—chorismate mutase and malate dehydrogenase—than the sequence generation methods from original studies specifically tailored for these families.”
From Large language models generate functional protein sequences across diverse families (2023) · cites this paper as [66]
On MDH function data, ProGen log-likelihoods had AUC 0.94, better than ProteinGAN discriminator scores at 0.87 (p<0.1, two-tailed, n=56).
“On MDH function data, ProGen log-likelihoods had an AUC of 0.94 (Fig. 4f), which was better than ProteinGAN discriminator scores, with an AUC of 0.87 (p<0.1, two-tailed test, n=56).”
From Large language models generate functional protein sequences across diverse families (2023) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Amino acid substitution matrices from protein blocks.
Henikoff, S., & Henikoff, J. G. (1992) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.89.22.10915 · Cited by 6,218 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
The Pfam protein families database in 2019
El-Gebali, S., Mistry, J., Bateman, A., Eddy, S. R., Luciani, A., Potter, S. C., Qureshi, M., Richardson, L. J., Salazar, G. A., Smart, A., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky995 · Cited by 4,602 · see citing works
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 2× · see citing works
“Sequence-only language models have since been trained through autoregressive or autoencoding self-supervision objectives for both discriminative and generative tasks, for example, using LSTMs or Transformer-based architectures [2, 9, 60, 64].”
MethodTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Jain, S., & Wallace, B. C. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1357 · Cited by 2,287 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1× · see citing works
“Methods that learn to explain by extracting rationales from the input and only using those rationales to make a prediction are faithful by design [8, 13, 43, 62].”
BackgroundUniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches
Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B., Wu, C. H., & the UniProt Consortium (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu739 · Cited by 1,928 · see citing works
A primer in BERTology: What we know about how BERT works
Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
What Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4828 · Cited by 1,514 · see citing works
The coming of age of de novo protein design
Huang, P.-S., Boyken, S. E., & Baker, D. (2016) · Nature · doi:10.1038/nature19946 · Cited by 1,478 · see citing works
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307 · see citing works
(2019) · doi:10.18653/v1/p19-1356 · Cited by 1,291 · see citing works
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1580 · Cited by 1,146 · see citing works
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131 · see citing works
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966 · see citing works
Clustering huge protein sequence sets in linear time
Steinegger, M., & Söding, J. (2018) · Nature Communications · doi:10.1038/s41467-018-04964-5 · Cited by 875 · see citing works
Ethayarajh, K. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1006 · Cited by 841 · see citing works
Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics
Asgari, E., & Mofrad, M. R. K. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141287 · Cited by 817 · see citing works
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · see citing works
Fox, N. K., Brenner, S. E., & Chandonia, J.-M. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1240 · Cited by 762 · see citing works
Evaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719 · see citing works
Arnold, F. H. (1998) · Accounts of Chemical Research · doi:10.1021/ar960017f · Cited by 706 · see citing works
Serrano, S., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1282 · Cited by 690 · see citing works
Linguistic Knowledge and Transferability of Contextual Representations
Liu, N. F., Gardner, M., Belinkov, Y., Peters, M. E., & Smith, N. A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1112 · Cited by 651 · see citing works
A Multiscale Visualization of Attention in the Transformer Model
Vig, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3007 · Cited by 600 · see citing works
Self-citationWhat you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties
Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1198 · Cited by 572 · see citing works
NetSurfP-2.0: improved prediction of protein structural features by integrated deep learning
Klausen, M., Jespersen, M. C., Nielsen, H., Jensen, K. K., Jurtz, V., Sønderby, C., Sommer, M., Winther, O., Nielsen, M., Petersen, B., et al. (2018) · bioRxiv · doi:10.1002/prot.25674 · Cited by 564 · see citing works
Revealing the Dark Secrets of BERT
Kovaleva, O., Romanov, A., Rogers, A., & Rumshisky, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1445 · Cited by 537 · see citing works
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models
Ettinger, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00298 · Cited by 511 · see citing works
NGL viewer: web-based molecular graphics for large complexes
Rose, A. S., Bradley, A. R., Valasatava, Y., Duarte, J. M., Prlić, A., & Rose, P. W. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty419 · Cited by 489 · see citing works
NGL Viewer: a web application for molecular visualization
Rose, A. S., & Hildebrand, P. W. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv402 · Cited by 462 · see citing works
HIV-1 protease: mechanism and drug discovery
Brik, A., & Wong, C. (2002) · Organic & Biomolecular Chemistry · doi:10.1039/b208248a · Cited by 456 · see citing works
Dissecting Contextual Word Embeddings: Architecture and Representation
Peters, M., Neumann, M., Zettlemoyer, L., & Yih, W.-T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1179 · Cited by 434 · see citing works
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
Rajani, N. F., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1487 · Cited by 421 · see citing works
Self-citationProbing Neural Network Comprehension of Natural Language Arguments
Niven, T., & Kao, H.-Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1459 · Cited by 395 · see citing works
What’s so special about model organisms?
Ankeny, R. A., & Leonelli, S. (2011) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2010.11.039 · Cited by 387 · see citing works
Assessing BERT's Syntactic Abilities
Goldberg, Y. (2019) · arXiv · doi:10.48550/arxiv.1901.05287 · Cited by 385 · see citing works
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint
Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04207 · Cited by 374 · see citing works
Measuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348 · see citing works
Critical assessment of methods of protein structure prediction (CASP)—Round XII
Moult, J., Fidelis, K. A., Kryshtafovych, A., Schwede, T. F., & Tramontano, A. L. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25415 · Cited by 327 · see citing works
Elnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 305 · see citing works
The european language resources association
Choukri, K., & Nilsson, M. (1998) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2ahosf78oz7c · Cited by 302 · see citing works
Analyzing the Structure of Attention in a Transformer Language Model
Vig, J., & Belinkov, Y. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4808 · Cited by 301 · see citing works
Self-citationVisualizing and Measuring the Geometry of BERTPreprint
Coenen, A., Reif, E., Yuan, A., Kim, B., Pearce, A., Viegas, F., & Wattenberg, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02715 · Cited by 298 · see citing works
ProGen: Language Modeling for Protein GenerationPreprint
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi, R. R., Huang, P., & Socher, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.07.982272 · Cited by 282 · see citing works
Self-citationAn Analysis of Encoder Representations in Transformer-Based Machine Translation
Raganato, A., & Tiedemann, J. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5431 · Cited by 277 · see citing works
Pathologies of Neural Models Make Interpretations Difficult
Feng, S., Wallace, E., Grissom, A., Iyyer, M., Rodríguez, P., & Boyd-Graber, J. L. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1407 · Cited by 260 · see citing works
Generative models for graph-based protein design
Ingraham, J., Garg, V., Barzilay, R., & Jaakkola, T. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 258 · see citing works
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint
Adi, Y. (2024) · arXiv (Cornell University) · doi:10.57702/tcr06vve · Cited by 237 · see citing works
Open Sesame: Getting inside BERT’s Linguistic Knowledge
Lin, Y., Tan, Y. C., & Frank, R. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4825 · Cited by 233 · see citing works
Learning protein sequence embeddings using information from structure
Bepler, T., & Berger, B. (2019) · arXiv · doi:10.48550/arxiv.1902.08661 · Cited by 219 · see citing works
Calibration of Pre-trained Transformers
Desai, S., & Durrett, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.21 · Cited by 209 · see citing works
BERT Rediscovers the Classical NLP PipelinePreprint
Tenney, I., Das, D., & Pavlick, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05950 · Cited by 192 · see citing works
A causal framework for explaining the predictions of black-box sequence-to-sequence models
Alvarez-Melis, D., & Jaakkola, T. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1042 · Cited by 185 · see citing works
Learning to Deceive with Attention-Based Explanations
Pruthi, D., Gupta, M., Dhingra, B., Neubig, G., & Lipton, Z. C. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.432 · Cited by 174 · see citing works
Re-thinking organisms: The impact of databases on model organism biology
Leonelli, S., & Ankeny, R. A. (2011) · Studies in History and Philosophy of Science Part C Studies in History and Philosophy of Biological and Biomedical Sciences · doi:10.1016/j.shpsc.2011.10.003 · Cited by 165 · see citing works
NGLview–interactive molecular graphics for Jupyter notebooks
Nguyen, H., Case, D. A., & Rose, A. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx789 · Cited by 164 · see citing works
Inferring protein 3D structure from deep mutation scans
Rollins, N. J., Brock, K. P., Poelwijk, F. J., Stiffler, M. A., Gauthier, N. P., Sander, C., & Marks, D. S. (2019) · Nature Genetics · doi:10.1038/s41588-019-0432-9 · Cited by 163 · see citing works
Attention Interpretability Across NLP TasksPreprint
Vashishth, S., Upadhyay, S., Tomar, G. S., & Faruqui, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11218 · Cited by 160 · see citing works
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models
Hoover, B., Strobelt, H., & Gehrmann, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.22 · Cited by 151 · see citing works
(2020) · neural information processing systems · Cited by 142 · see citing works
ProteinNet: a standardized data set for machine learning of protein structure
AlQuraishi, M. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2932-0 · Cited by 141 · see citing works
WT5?! Training Text-to-Text Models to Explain their PredictionsPreprint
Narang, S., Raffel, C., Lee, K., Roberts, A., Fiedel, N., & Malkan, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14546 · Cited by 141 · see citing works
Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized EmbeddingsPreprint
Wiedemann, G., Remus, S., Chawla, A., & Biemann, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10430 · Cited by 130 · see citing works
Do Attention Heads in BERT Track Syntactic Dependencies?Preprint
Htut, P. M., Phang, J., Bordia, S., & Bowman, S. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.12246 · Cited by 122 · see citing works
What do you mean, BERT? Assessing BERT as a Distributional Semantics ModelPreprint
Mickus, T., Paperno, D., Constant, M., & van Deemter, K. (2019) · arXiv (Cornell University) · doi:10.7275/t778-ja71 · Cited by 74 · see citing works
Grammar of protein domain architectures
Yu, L., Tanwar, D. K., Penha, E. D. D. S., Wolf, Y. I., Koonin, E. V., & Basu, M. K. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1814684116 · Cited by 68 · see citing works
Assessing Social and Intersectional Biases in Contextualized Word RepresentationsPreprint
Tan, Y. C., & Celis, L. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01485 · Cited by 67 · see citing works
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender BiasPreprint
Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang, J., Singer, Y., & Shieber, S. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12265 · Cited by 63 · see citing works
Self-citationOn Identifiability in Transformers
Brunner, G., Liu, Y., Pascual, D., Richter, O., Ciaramita, M., & Wattenhofer, R. P. (2020) · International Conference on Learning Representations · Cited by 62 · see citing works
What do you mean, BERT? Assessing BERT as a Distributional Semantics ModelPreprint
Mickus, T., Paperno, D., Constant, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05758 · Cited by 61 · see citing works
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers ModelsPreprint
Hoover, B., Strobelt, H., & Gehrmann, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05276 · Cited by 55 · see citing works
Low-N protein engineering with data-efficient deep learningPreprint
Biswas, S., Khimulya, G., Alley, E. C., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.01.23.917682 · Cited by 54 · see citing works
Unified rational protein engineering with sequence-only deep representation learningPreprint
Alley, E. C., Khimulya, G., Biswas, S., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/589333 · Cited by 50 · see citing works
Interpretable Neural Predictions with Differentiable Binary Variables
Bastings, J., Aziz, W. F., & Titov, I. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1284 · Cited by 47 · see citing works
Comprehensive Structural Classification of Ligand-Binding Motifs in Proteins
Kinjo, A. R., & Nakamura, H. (2009) · Structure · doi:10.1016/j.str.2008.11.009 · Cited by 45 · see citing works
ProGen: Language Modeling for Protein GenerationPreprint
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi, R. R., Huang, P., & Socher, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03497 · Cited by 42 · see citing works
Self-citationProtein Engineering by Combined Computational and In Vitro Evolution Approaches
Rosenfeld, L., Heyne, M., Shifman, J. M., & Papo, N. (2016) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2016.03.002 · Cited by 41 · see citing works
A Game Theoretic Approach to Class-wise Selective RationalizationPreprint
Chang, S., Zhang, Y., Yu, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12853 · Cited by 39 · see citing works
Fine-grained Sentiment Analysis with Faithful AttentionPreprint
Zhong, R., Shao, S., & McKeown, K. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06870 · Cited by 37 · see citing works
Diagnostic Classifiers: Revealing how Neural Networks Process Hierarchical Structure
Veldhoen, S., Hupkes, D., & Zuidema, W. H. (2016) · UvA-DARE (University of Amsterdam) · Cited by 36 · see citing works
Adaptively Sparse TransformersPreprint
Correia, G. M., Niculae, V., & Martins, A. F. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00015 · Cited by 35 · see citing works
Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models.
Grimsley, C., Mayfield, E., & Bursten, J. R. S. (2020) · PhilPapers (PhilPapers Foundation) · Cited by 32 · see citing works
Unsupervised Attention-Guided Atom-MappingPreprint
Schwaller, P., Hoover, B., Reymond, J., et al. (2020) · ChemRxiv · doi:10.26434/chemrxiv.12298559.v1 · Cited by 26 · see citing works
Pretrained AI Models: Performativity, Mobility, and Change
Varshney, L. R., Keskar, N. S., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.03290 · Cited by 20 · see citing works
Self-citationBERT Learns (and Teaches) ChemistryPreprint
Payne, J., Srouji, M. S., Yap, D. A., & Kosaraju, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.16012 · Cited by 19 · see citing works
Interrogating the Explanatory Power of Attention in Neural Machine Translation
Moradi, P., Kambhatla, N., & Sarkar, A. (2019) · Workshop on Neural Generation and Translation (WNGT) · doi:10.18653/v1/d19-5624 · Cited by 18 · see citing works
Open Sesame: Getting Inside BERT's Linguistic KnowledgePreprint
Lin, Y., Tan, Y. C., & Frank, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01698 · Cited by 15 · see citing works
Protein engineering: security implications
Tucker, J. Β., & Hooper, C. (2006) · EMBO Reports · doi:10.1038/sj.embor.7400677 · Cited by 13 · see citing works
Lee, J., Konc, J., Janežič, D., & Brooks, B. R. (2017) · Scientific Reports · doi:10.1038/s41598-017-10412-z · Cited by 7 · see citing works
EXS: Explainable Search Using Local Model Agnostic InterpretabilityPreprint
Singh, J., & Anand, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03857 · Cited by 3 · see citing works
ProteinNet: a standardized data set for machine learning of protein structurePreprint
AlQuraishi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00249 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Hongge Chen, Si Si, Yang Li, Ciprian Chelba, Sanjiv Kumar, Duane S. Boning, and Cho-Jui Hsieh. Multi-stage influence function. Preprint, 2019.
- Sara El-Gebali, Jaina Mistry, Alex Bateman, Sean R Eddy, Aurélien Luciani, Simon C Potter, Matloob Qureshi, Lorna J Richardson, Gustavo A Salazar, Alfredo Smart, Erik L L Sonnhammer, Layla Hirsh, Lisanna Paladin, Damiano Piovesan, Silvio C E Tosatto, and Robert D Finn. The Pfam protein families database in 2019. Nucleic Acids Research, 47(D1):D427-D432, 2019. ISSN 0305-1048. doi: 10.1093/nar/gky995. URL https://academic.oup.com/nar/ article/47/D1/D427/5144153. · DOI · Paper page
- Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. What does BERT learn about the structure of language? In ACL 2019 -57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019. URL https://hal.inria.fr/hal-02131630.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Dis- tributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 3111- 3119.
- Adam J Riesselman, Jung-Eun Shin, Aaron W Kollasch, Conor McMahon, Elana Simon, Chris Sander, Aashish Manglik, Andrew C Kruse, and Debora S Marks. Accelerating protein design using autoregressive generative models. bioRxiv, page 757252, 2019.
- D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg. SmoothGrad: removing noise by adding noise. ICML workshop on visualization for deep learning, June 2017.
- Haizi Yu, Lav R. Varshney, and Genevieve Stein-O'Brien. Towards learning human-interpretable laws of neurogenesis from single-cell RNA-seq data via information lattices. In Learning Meaningful Representations of Life Workshop at NeurIPS 2019, December 2019.
Cited by
Sorted by citations
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Large language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01618-2 · Cited by 1,078 · see citing works
Self-citationLanguage models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.07.09.450648 · Cited by 635 · see citing works
Rethinking Attention with Performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · see citing works
“Furthermore, we analyze the amino-acid similarity matrix estimated from the attention matrices produced by the Performer model, as described in Vig et al. (Vig et al., 2020).”
MethodNucleotide Transformer: building and evaluating robust foundation models for human genomics
Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski, A. H., Oteri, F., Dallago, C., Trop, E., de Almeida, B. P., Sirelkhatim, H., et al. (2025) · Nature Methods · doi:10.1038/s41592-024-02523-z · Cited by 656 · see citing works
“We followed a methodology proposed in previous work 38 .”
InfluentialLarge-scale chemical language representations capture molecular structure and properties
Ross, J., Belgodere, B., Chenthamarakshan, V., Padhi, I., Mroueh, Y., & Das, P. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00580-7 · Cited by 590 · see citing works
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · see citing works
High-resolution <i>de novo</i> structure prediction from primary sequence
Wu, R., Ding, F., Wang, R., Shen, R., Zhang, X., Luo, S., Su, C., Wu, Z., Xie, Q., Berger, B., et al. (2022) · doi:10.1101/2022.07.21.500999 · Cited by 392 · see citing works
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341 · see citing works
Graph representation learning in biomedicine and healthcare
Li, M. M., Huang, K., & Zitnik, M. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00942-x · Cited by 294 · see citing works
“Additionally, an attention mechanism for protein-sequence embeddings generated by the language model BERT (for ‘bidirectional encoder representations from transformers’) has facilitated the interpretability of the predictions of such networks 129,130 .”
Antibody structure prediction using interpretable deep learning
Ruffolo, J. A., Sulam, J., & Gray, J. J. (2022) · Patterns · doi:10.1016/j.patter.2021.100406 · Cited by 216 · see citing works
Prihoda, D., Maamary, J., Waight, A., Juan, V., Fayadat-Dilman, L., Svozil, D., & Bitton, D. A. (2022) · mAbs · doi:10.1080/19420862.2021.2020203 · Cited by 190 · see citing works
Language models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181 · see citing works
Convolutions are competitive with transformers for protein sequence pretraining
Yang, K. K., Fusi, N., & Lu, A. X. (2024) · Cell Systems · doi:10.1016/j.cels.2024.01.008 · Cited by 137 · see citing works
Structure-based protein design with deep learning
Ovchinnikov, S., & Huang, P.-S. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.08.004 · Cited by 93 · see citing works
Genomic language model predicts protein co-regulation and function
Hwang, Y., Cornman, A. L., Kellogg, E. H., Ovchinnikov, S., & Girguis, P. R. (2024) · Nature Communications · doi:10.1038/s41467-024-46947-9 · Cited by 99 · see citing works
“Subsequently, different attention heads in pLMs 40 have been shown to correlate to speci fi c structural elements and functional sites in a protein.”
Novel machine learning approaches revolutionize protein knowledge
Bordin, N., Dallago, C., Heinzinger, M., Kim, S., Littmann, M., Rauer, C., Steinegger, M., Rost, B., & Orengo, C. (2023) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2022.11.001 · Cited by 72 · see citing works
GENA-LM: A Family of Open-Source Foundational DNA Language Models for Long Sequences
Fishman, V., Kuratov, Y., Shmelev, A., Petrov, M., Penzar, D., Shepelin, D., Chekanov, N., Kardymon, O., & Burtsev, M. (2023) · doi:10.1101/2023.06.12.544594 · Cited by 137 · see citing works
“While preliminary layers prioritize local level representations, the advanced layers capture intricate global features, such as binding sites and contact maps [42].”
Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models
Li, F.-Z., Amini, A. P., Yue, Y., Yang, K. K., & Lu, A. X. (2024) · doi:10.1101/2024.02.05.578959 · Cited by 51 · see citing works
- Authors
- Vig, J., Madani, A., Varshney, L. R., Xiong, C., Socher, R., & Rajani, N. F.
- Affiliations
- Salesforce (United States) (US)
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Vig, Jesse
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2020-06-28
- OpenAlex
- W3037620288
- Cited by
- 338
- References
- 104
- Citations by year
- 2026: 40 · 2025: 60 · 2024: 64 · 2023: 49 · 2022: 64 · 2021: 53 · 2020: 8
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Learning Representations
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Keywords
- protein language models, attention mechanism, interpretability, protein folding, protein binding sites, biophysical properties, transformer models, protein structure, attention visualization, protein engineering, synthetic biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
