Protein language models are biased by unequal sequence sampling across the tree of life
Open accessAbstract
Protein language models (pLMs) trained on large protein sequence databases have been used to understand disease and design novel proteins. In design tasks, the likelihood of a protein sequence under a pLM is often used as a proxy for protein fitness, so it is critical to understand what signals likelihoods capture. In this work we find that pLM likelihoods unintentionally encode a species bias: likelihoods of protein sequences from certain species are systematically higher, independent of the protein in question. We quantify this bias and show that it arises in large part because of unequal species representation in popular protein sequence databases. We further show that the bias can be detrimental for some protein design applications, such as enhancing thermostability. These results highlight the importance of understanding and curating pLM training data to mitigate biases and improve protein design capabilities in under-explored parts of sequence space.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data diversity and quality are critical to ensure model performance.
“In addition to data size, data diversity and quality are critical to ensure model performance.114”
From How to build the virtual cell with artificial intelligence: Priorities and opportunities (2024) · cites this paper as [114]
The Introduction says under-represented groups can have lower model accuracy and that increasing dataset diversity can improve overall performance. The Conclusion emphasizes data curation for protein language models, but it does not establish data quality as a separate determinant of model performance or show that diversity and quality ensure performance. (LLM, conf=0.82)
Experimental — accuracy not yet measuredData from humans and model organisms such as mice and Escherichia coli are unequally represented in sequence and literature databases, which when used for training encode strong species biases.
“Data from humans and model organisms, such as mice and Escherichia coli, are unequally represented in sequence and literature databases, which when used for training, encode strong species biases.114”
From How to build the virtual cell with artificial intelligence: Priorities and opportunities (2024) · cites this paper as [114]
The Introduction and Section 5 link unequal representation in protein sequence databases to species bias in protein language models. Section 5 identifies humans, mice, and E. coli as species with high sequence counts and high model ratings. The paper does not establish the corresponding claim about literature databases or models trained on them. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
UniProt: the Universal Protein Knowledgebase in 2023
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., Bye-A-Jee, H., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkac1052 · Cited by 7,210 · see citing works
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 2× · see citing works
“…processing, many different protein language models (pLMs) have been trained to model the distribution of naturally occurring protein sequences (Rives et al., 2021; Elnaggar et al., 2021; Madani et al., 2023; Lin et al., 2023; Alamdari et al., 2023). pLMs have been successfully used to predict…”
MethodWatson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., et al. (2022) · doi:10.1101/2022.12.09.519842 · Cited by 2,369 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · see citing works
De novo design of protein structure and function with RFdiffusion
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., et al. (2023) · Nature · doi:10.1038/s41586-023-06415-8 · Cited by 2,325 · see citing works
Robust deep learning based protein sequence design using ProteinMPNN
Dauparas, J., Anishchenko, I., Bennett, N., Bai, H., Ragotte, R. J., Milles, L. F., Wicky, B. I. M., Courbet, A., de Haas, R. J., Bethel, N., et al. (2022) · doi:10.1101/2022.06.03.494563 · Cited by 2,107 · see citing works
Robust deep learning–based protein sequence design using ProteinMPNN
Dauparas, J., Anishchenko, I., Bennett, N., Bai, H., Ragotte, R. J., Milles, L. F., Wicky, B. I. M., Courbet, A., de Haas, R. J., Bethel, N., et al. (2022) · Science · doi:10.1126/science.add2187 · Cited by 2,070 · see citing works
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · see citing works
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131 · cited 1× · see citing works
“…al., 2022; Ferruz et al., 2022; Elnaggar et al., 2023; Lin et al., 2023; Nijkamp et al., 2023), CNNs (Yang et al., 2022), and other architectures (Alley et al., 2019). pLMs can generate sequences that successfully fold into functional proteins, both by sampling from the model unconditionally,…”
BackgroundLarge language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01618-2 · Cited by 1,078 · cited 2× · see citing works
“…models (pLMs) have been trained to model the distribution of naturally occurring protein sequences (Rives et al., 2021; Elnaggar et al., 2021; Madani et al., 2023; Lin et al., 2023; Alamdari et al., 2023). pLMs have been successfully used to predict protein 3D structure (Lin et al., 2023),…”
BackgroundMethodMen Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1× · see citing works
“Various papers have studied how re-weighting or curating datasets can mitigate these biases (Zhao et al., 2017; Ryu et al., 2017; Tschandl et al., 2018; Yang et al., 2020), even finding that overall performance is improved by over-weighting minority groups and actively increasing diversity in…”
BackgroundProGen2: Exploring the boundaries of protein language models
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.002 · Cited by 480 · see citing works
Deduplicating Training Data Makes Language Models Better
Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.577 · Cited by 304 · see citing works
Yang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375709 · Cited by 254 · see citing works
Language models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181 · cited 1× · see citing works
“…et al., 2022), and other architectures (Alley et al., 2019). pLMs can generate sequences that successfully fold into functional proteins, both by sampling from the model unconditionally, and sampling conditionally on some inputs (Verkuil et al., 2022; Madani et al., 2023; Alamdari et al., 2023).”
BackgroundHumanization of antibodies using a machine learning approach on large-scale repertoire data
Marks, C., Hummer, A. M., Chin, M., & Deane, C. M. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab434 · Cited by 124 · see citing works
Zhu, D., Brookes, D. H., Busia, A., Carneiro, A., Fannjiang, C., Popova, G., Shin, D., Donohue, K. C., Lin, L. F., Miller, Z. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adj3786 · Cited by 63 · see citing works
Flagged first, then by citations
Research works printed but not linked: 42
- Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Alex Xijie Lu, Nicolo Fusi, Ava Pardis Amini, and Kevin K Yang. Protein generation with evolutionary diffusion: sequence is all you need. bioRxiv, pp. 2023-09, 2023.
- Amos Bairoch and Rolf Apweiler. The swiss-prot protein sequence database and its supplement trembl in 2000. Nucleic acids research, 28(1):45-48, 2000.
- Meriem Boubdir, Edward Kim, Beyza Ermis, Sara Hooker, and Marzieh Fadaee. Elo uncovered: Robustness and best practices in language model evaluation. arXiv preprint arXiv:2311.17295, 2023. · arXiv
- Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial. Proteinbert: a universal deep-learning model of protein sequence and function. Bioinformatics, 38(8):2102-2110, 2022.
- Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pp. 77-91. PMLR, 2018.
- Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183-186, 2017.
- Irene Chen, Fredrik D Johansson, and David Sontag. Why is my classifier discriminatory? Advances in neural information processing systems, 31, 2018.
- Lobna Daoud and Mamdouh Ben Ali. Chapter 5 -halophilic microorganisms: Interesting group of extremophiles with important applications in biotechnology and environment. In Richa Salwan and Vivek Sharma (eds.), Physiological and Biotechnological Aspects of Extremophiles, pp. 51-64. Academic Press, 2020. ISBN 978-0-12-818322-9. doi: https://doi.org/10.1016/B978-0-12-818322-9.00005-8. URL https://www.sciencedirect.com/science/article/pii/B9780128183229000058. · DOI
- Ahmed Elnaggar, Hazem Essam, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, and Burkhard Rost. Ankh: Optimized protein language model unlocks general-purpose modelling. bioRxiv, pp. 2023-01, 2023.
- Arpad E Elo. The Rating of Chess Players: Past and Present. Ishi Press International, 1978.
- Hyunuk Eom, Kye Soo Cho, Jihyeon Lee, Stephanie Kim, Sukhwan Park, Hyunbin Kim, Jinsol Yang, Young-Hyun Han, Juyong Lee, Chaok Seok, et al. Discovery of highly active kynureninases for cancer immunotherapy through protein language model. bioRxiv, pp. 2024-01, 2024.
- Clara Fannjiang, Micah Olivas, Eric R Greene, Craig J Markin, Bram Wallace, Ben Krause, Margaux M Pinney, James Fraser, Polly M Fordyce, Ali Madani, et al. Designing active and thermostable enzymes with sequence-only predictive models. In NeurIPS 2022 Workshop on Learning Meaningful Representations of Life, 2022.
- Noelia Ferruz, Steffen Schmidt, and Birte Höcker. Protgpt2 is a deep unsupervised language model for protein design. Nature communications, 13(1):4348, 2022.
- Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. · arXiv
- Nina Gunde-Cimerman, Ana Plemenitaš, and Aharon Oren. Strategies of adaptation of microorganisms of the three domains of life to high salt concentrations. FEMS microbiology reviews, 42(3):353-375, 2018.
- Daniel Hesslow, Niccoló Zanichelli, Pascal Notin, Iacopo Poli, and Debora Marks. Rita: a study on scaling up generative protein sequence models. arXiv preprint arXiv:2205.05789, 2022. · arXiv
- Chloe Hsu, Hunter Nisonoff, Clara Fannjiang, and Jennifer Listgarten. Learning protein fitness models from evolutionary and assay-labeled data. Nature biotechnology, 40(7):1114-1122, 2022a.
- Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 8946-8970. PMLR, 17-23 Jul 2022b. URL https://proceedings.mlr.press/v162/hsu22a.html.
- John B Ingraham, Max Baranov, Zak Costello, Karl W Barber, Wujie Wang, Ahmed Ismail, Vincent Frappier, Dana M Lord, Christopher Ng-Thow-Hing, Erik R Van Vlack, et al. Illuminating protein space with a programmable generative model. Nature, pp. 1-9, 2023.
- Milind Jagota, Chengzhong Ye, Carlos Albors, Ruchir Rastogi, Antoine Koehl, Nilah Ioannidis, and Yun S Song. Cross-protein transfer learning substantially improves disease variant prediction. Genome Biology, 24(1):182, 2023.
- Sean R Johnson, Xiaozhi Fu, Sandra Viknander, Clara Goldin, Sarah Monaco, Aleksej Zelezniak, and Kevin K Yang. Computational scoring and experimental evaluation of enzymes generated by neural networks. bioRxiv, pp. 2023-03, 2023.
- Felix Jung, Kevin Frey, David Zimmer, and Timo Mühlhaus. Deepstabp: A deep learning approach for the prediction of thermal protein stability. International Journal of Molecular Sciences, 24(8):7444, 2023.
- Giona Kleinberg, Michael J Diaz, Sai Batchu, and Brandon Lucke-Wold. Racial underrepresentation in dermatological datasets leads to biased machine learning models and inequitable healthcare. Journal of biomed research, 3(1):42, 2022.
- Lukasz P Kozlowski. Ipc-isoelectric point calculator. Biology direct, 11(1):1-16, 2016.
- Chu-Cheng Lin, Aaron Jaech, Xin Li, Matthew R. Gormley, and Jason Eisner. Limitations of autoregressive models and their alternatives. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani- Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5147-5173, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.naacl-main.405. URL https://aclanthology.org/2021.naacl-main.405. · DOI
- R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638, 2023. · arXiv
- Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in Neural Information Processing Systems, 34:29287-29303, 2021.
- Roni Miah, Ayesha Siddiqa, Udvashita Chakraborty, Jamsheda Ferdous Tuli, Noyon Kumar Barman, Aukhil Uddin, Tareque Aziz, Nadim Sharif, Shuvra Kanti Dey, Mamoru Yamada, et al. Development of high temperature simultaneous saccharification and fermentation by thermosensitive saccharomyces cerevisiae and bacillus amyloliquefaciens. Scientific Reports, 12(1):3630, 2022.
- H Pezeshgi Modarres, MR Mofrad, and AJRA Sanati-Nezhad. Protein thermostability engineering. RSC advances, 6(116):115252-115270, 2016.
- Pascal Notin, Aaron W Kollasch, Daniel Ritter, Lood Van Niekerk, Steffan Paul, Han Spinner, Nathan J Rollins, Ada Shaw, Rose Orenbuch, Ruben Weitzman, et al. Proteingym: Large-scale benchmarks for protein fitness prediction and design. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023a.
- Pascal Notin, Ruben Weitzman, Debora Susan Marks, and Yarin Gal. Proteinnpt: Improving protein property prediction and design with non-parametric transformers. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b.
- Pedro A Ortega, Markus Kunesch, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Joel Veness, Jonas Buchli, Jonas Degrave, Bilal Piot, Julien Perolat, et al. Shaking the foundations: delusions in sequence models for interaction and control. arXiv preprint arXiv:2110.10819, 2021. · arXiv
- Esther Rolf, Theodora T Worledge, Benjamin Recht, and Michael Jordan. Representation matters: Assessing the importance of subgroup allocations in training data. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 9040-9051. PMLR, 18-24 Jul 2021. URL https://proceedings.mlr.press/v139/rolf21a. html.
- Hee Jung Ryu, Hartwig Adam, and Margaret Mitchell. Inclusivefacenet: Improving face attribute detection with race and gender diversity. arXiv preprint arXiv:1712.00193, 2017. · arXiv
- Nima Shahbazi, Yin Lin, Abolfazl Asudeh, and HV Jagadish. Representation bias in data: A survey on identification and resolution techniques. ACM Computing Surveys, 2023.
- Ada Y Shaw, Hansen B Spinner, Sarah Gurev, Jung-Eun Shin, Nathan Rollins, and Debora S Marks. Removing bias in sequence models of protein fitness. bioRxiv, pp. 2023-09, 2023.
- Jung-Eun Shin, Adam J Riesselman, Aaron W Kollasch, Conor McMahon, Elana Simon, Chris Sander, Aashish Manglik, Andrew C Kruse, and Debora S Marks. Protein design and variant prediction using autoregressive generative models. Nature communications, 12(1):2403, 2021.
- Alexey Strokach, David Becerra, Carles Corbi-Verge, Albert Perez-Riba, and Philip M. Kim. Fast and flexible protein design using deep graph neural networks. Cell Systems, 11(4):402-411.e4, 2020. ISSN 2405-4712. doi: https://doi.org/10.1016/j.cels.2020.08.016. URL https://www.sciencedirect.com/science/article/ pii/S2405471220303276. · DOI
- Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consortium. Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics, 31(6):926-932, 2015.
- Rohan Taori and Tatsunori Hashimoto. Data feedback loops: Model-driven amplification of dataset biases. In International Conference on Machine Learning, pp. 33883-33920. PMLR, 2023.
- Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
- Kevin K Yang, Nicolo Fusi, and Alex X Lu. Convolutions are competitive with transformers for protein sequence pretraining. bioRxiv, pp. 2022-05, 2022.
Cited by
Sorted by citations
How to build the virtual cell with artificial intelligence: Priorities and opportunities
Bunne, C., Roohani, Y., Rosen, Y., Gupta, A., Zhang, X., Roed, M., Alexandrov, T., AlQuraishi, M., Brennan, P., Burkhardt, D. B., et al. (2024) · Cell · doi:10.1016/j.cell.2024.11.015 · Cited by 393 · see citing works
“114 Other biases, for example, in terms of sex, specific diseases, or human ancestral populations could also reduce the impact of AIVC models.”
A Vast Navigable Protein Sequence Subspace
Sinai, S., Gane, A., Jain, N., Bryant, D., Bileschi, M., Bashir, A., Church, G. M., Kelsic, E., & Colwell, L. (2026) · doi:10.64898/2026.10.18.759989
- Authors
- Ding, F., & Steinhardt, J.
- Affiliations
- University of California System (US)
- Corresponding author
- Steinhardt, Jacob
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2024-03-12
- OpenAlex
- W4392686654
- Cited by
- 57
- References
- 19
- Citations by year
- 2026: 18 · 2025: 25 · 2024: 14
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by
- Locations
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Funders
- National Science Foundation (1752814, 1804794)
- Open Philanthropy Project
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, protein sequence databases, protein design, protein thermal stability, training data curation, tree of life
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
