Contrastive learning in protein language space predicts interactions between drugs and protein targets
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Sequence-based prediction of drug–target interactions has the potential to accelerate drug discovery by complementing experimental screens. Such computational prediction needs to be generalizable and scalable while remaining sensitive to subtle variations in the inputs. However, current computational techniques fail to simultaneously meet these goals, often sacrificing performance of one to achieve the others. We develop a deep learning model, ConPLex, successfully leveraging the advances in pretrained protein language models (“PLex”) and employing a protein-anchored contrastive coembedding (“Con”) to outperform state-of-the-art approaches. ConPLex achieves high accuracy, broad adaptivity to unseen data, and specificity against decoy compounds. It makes predictions of binding based on the distance between learned representations, enabling predictions at the scale of massive compound libraries and the human proteome. Experimental testing of 19 kinase-drug interaction predictions validated 12 interactions, including four with subnanomolar affinity, plus a strongly binding EPHB1 inhibitor ( K D = 1.3 nM). Furthermore, ConPLex embeddings are interpretable, which enables us to visualize the drug–target embedding space and use embeddings to characterize the function of human cell-surface proteins. We anticipate that ConPLex will facilitate efficient drug discovery by making highly sensitive in silico drug screening feasible at the genome scale. ConPLex is available open source at https://ConPLex.csail.mit.edu .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
DrugBank 5.0: a major update to the DrugBank database for 2018
Wishart, D. S., Feunang, Y. D., Guo, A. C., Lo, E. J., Marcu, A., Grant, J. R., Sajed, T., Johnson, D., Li, C., Sayeeda, Z., et al. (2018) · Nucleic Acids Research · doi:10.1093/nar/gkx1037 · Cited by 8,404 · cited 3×
“Wolff et al. (43) previously identified PD-166326 as a tyrosinekinase inhibitor but did not report any binding to EPHB1, and DrugBank (44) lists only ABL1 as a known target (DrugBank ID: DB08339).”
InfluentialAccelerated Profile HMM Searches
Eddy, S. R. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002195 · Cited by 6,975 · cited 3×
“We identified Pfam domains (50) for each protein in the Surfaceome database using HMMscan (51) and compared the projections of proteins that share the same domains.”
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
The Pfam protein families database in 2019
El-Gebali, S., Mistry, J., Bateman, A., Eddy, S. R., Luciani, A., Potter, S. C., Qureshi, M., Richardson, L. J., Salazar, G. A., Smart, A., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky995 · Cited by 4,602 · cited 3×
“We identified Pfam domains (50) for each protein in the Surfaceome database using HMMscan (51) and compared the projections of proteins that share the same domains.”
ZINC − A Free Database of Commercially Available Compounds for Virtual Screening
Irwin, J. J., & Shoichet, B. K. (2005) · Journal of Chemical Information and Modeling · doi:10.1021/ci049714+ · Cited by 4,058 · cited 3×
“We selected 51 kinases from the Surfaceome database (17) with commercially available assays from the DiscoveryX company and used ConPLex to scan against a set of 4715 compounds from the ZINC database (37) purchasable from the Cayman Chemical Company (Methods) (38).”
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 1×
“Specifically, we investigate the pretrained models Prose (30), ESM (76), and ProtBert (27), with default dimensions dt = 6165, 1280, and1024, respectively (SI Appendix, S2).”
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking.
Mysinger, M. M., Carchia, M., Irwin, J. J., & Shoichet, B. K. (2012) · Journal of Medicinal Chemistry · doi:10.1021/jm300687e · Cited by 2,412 · cited 5×
“Parameters are updated using the triplet distance loss on a high-coverage dataset (36) to minimize the target-drug distance while maximizing the target-decoy distance.”
InfluentialComprehensive analysis of kinase inhibitor selectivity
Davis, M. I., Hunt, J. P., Herrgard, S., Ciceri, P., Wodicka, L. M., Pallares, G., Hocker, M., Treiber, D. K., & Zarrinkar, P. P. (2011) · Nature Biotechnology · doi:10.1038/nbt.1990 · Cited by 2,409 · cited 4×
“Two datasets, DAVIS (68) and BindingDB (69), consist of pairs of drugs and targets with experimentally determined dissociation constants (KD).”
InfluentialChEMBL: towards direct deposition of bioassay data
Mendez, D., Gaulton, A., Bento, A. P., Chambers, J., De Veij, M., Félix, E., Magariños, M. P., Mosquera, J. F., Mutowo, P., Nowotka, M., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky1075 · Cited by 2,372
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103
Estimated Research and Development Investment Needed to Bring a New Medicine to Market, 2009-2018
Wouters, O. J., McKee, M., & Luyten, J. (2020) · JAMA · doi:10.1001/jama.2020.1166 · Cited by 1,531 · cited 1×
“Drug discovery is a fundamental task for human health yet remains both extremely expensive and time consuming, with the median drug requiring over 1 billion dollars (65) and 10 y (66) from development to approval and distribution.”
Tsubaki, M., Tomii, K., & Sese, J. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty535 · Cited by 688 · cited 2×
“Here, we compare with several methods which use non-PLM protein features: MolTrans (13), GNNCPI (34), and DeepConv-DTI (12).”
Modeling the language of life – Deep Learning Protein Sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · doi:10.1101/614313 · Cited by 606
Modeling aspects of the language of life through transfer-learning protein sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3220-8 · Cited by 606 · cited 1×
“As a method to use pretrained PLMs for DTI prediction, ConPLex is yet another example of the power of transferring learned representations for biology (13, 31, 32, 56, 57).”
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · cited 1×
“Specifically, we investigate the pretrained models Prose (30), ESM (76), and ProtBert (27), with default dimensions dt = 6165, 1280, and1024, respectively (SI Appendix, S2).”
Self-citationHigh-resolution de novo structure prediction from primary sequence
Wu, R., Ding, F., Wang, R., Shen, R., Zhang, X., Luo, S., Su, C., Wu, Z., Xie, Q., Berger, B., et al. (2022) · doi:10.1101/2022.07.21.500999 · Cited by 392 · cited 1×
“While structural similarity is often implicitly learned by PLMs, future work could explicitly incorporate structure where such data are available, perhaps by incorporating a more advanced projection architecture like the Geoformer (3).”
Self-citationDrugs, Devices, and the FDA: Part 1
Van Norman, G. A. (2016) · JACC: Basic to Translational Science · doi:10.1016/j.jacbts.2016.03.002 · Cited by 366
Learning the language of viral evolution and escape
Hie, B., Zhong, E. D., Berger, B., & Bryson, B. (2021) · Science · doi:10.1126/science.abd7331 · Cited by 361 · cited 1×
“As a method to use pretrained PLMs for DTI prediction, ConPLex is yet another example of the power of transferring learned representations for biology (13, 31, 32, 56, 57).”
Self-citationLearning protein sequence embeddings using information from structure
Bepler, T., & Berger, B. (2019) · arXiv · doi:10.48550/arxiv.1902.08661 · Cited by 219
Self-citationPanoramic view of a superfamily of phosphatases through substrate profiling
Huang, H., Pandya, C., Liu, C., Al-Obaidi, N. F., Wang, M., Zheng, L., Toews Keating, S., Aono, M., Love, J. D., Evans, B., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1423570112 · Cited by 168 · cited 2×
“These include DTI data on β -ketoacid cleavage (BKACE) (75), Esterase (72), Glycosyltransferases (73), Halogenase (74), and Phosphatase (71) enzymes.”
Leveraging Uncertainty in Machine Learning Accelerates Biological Discovery and Design
Hie, B., Bryson, B. D., & Berger, B. (2020) · Cell Systems · doi:10.1016/j.cels.2020.09.007 · Cited by 154
Self-citationMachine learning modeling of family wide enzyme-substrate specificity screens
Goldman, S., Das, R., Yang, K. K., & Coley, C. W. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009853 · Cited by 120
Realizing private and practical pharmacological collaboration
Hie, B., Cho, H., & Berger, B. (2018) · Science · doi:10.1126/science.aat4807 · Cited by 76
Self-citation
Research works printed but not linked: 57
- 10.1126/science.abj8754 · doi:10.1126/science.abj8754 · Paper page
- 10.3390/ijms20184331 · DOI · Paper page
- 10.1002/bit.26737 · DOI
- 10.1093/bioinformatics/btq595 · DOI
- S. J. Trudeau PrePCI: A structure- and chemical similarity-informed database of predicted protein compound interactions. bioRxiv [Preprint] (2022). https://doi.org/10.1101/2022.09.17.508184 (Accessed 7 December 2022). · DOI · Paper page
- 10.1093/nar/gkq481 · DOI · Paper page
- E. Anderson, G. D. Veith, D. Weininger, SMILES, A Line Notation and Computerized Interpreter for Chemical Structures (Environmental Research Laboratory, US Environmental Protection Agency, 1987).
- 10.1093/bib/bbz157 · DOI · Paper page
- 10.1371/journal.pcbi.1007129 · DOI · Paper page
- 10.1093/bioinformatics/btaa880 · DOI · Paper page
- S. Sledzieski R. Singh L. Cowen B. Berger. “Adapting protein language models for rapid DTI prediction in Machine Learning for Structural Biology Workshop (MLSB) at NeurIPS (2021). · DOI
- 10.1073/pnas.1808790115 · DOI · Paper page
- K. Huang Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv [Preprint] (2021). http://arxiv.org/abs/2102.09548 (Accessed 7 December 2022). · arXiv
- 10.1093/bib/bbac199 · DOI
- 10.1111/acel.12819 · DOI · Paper page
- 10.1073/pnas.2025581118 · DOI · Paper page
- 10.1093/bioinformatics/btaa1005 · DOI · Paper page
- 10.1038/s41598-019-54180-4 · DOI
- 10.1021/c160017a018 · DOI · Paper page
- A. Elnaggar ProtTrans: Towards cracking the language of life’s code through self-supervised deep learning and high performance computing. arXiv [Preprint] (2020). http://arxiv.org/abs/2007.06225 (Accessed 7 December 2022). · arXiv
- 10.1016/j.gde.2015.08.010 · DOI
- S. Sledzieski, R. Singh, L. Cowen, B. Berger, D-SCRIPT translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein–protein interactions. Cell Syst. 12, 1–14 (2021).
- 10.1093/bioinformatics/btac258 · DOI · Paper page
- 10.1093/nargab/lqac043 · doi:10.1093/nargab/lqac043
- 10.1021/ci049714 · DOI
- 10.1139/y2012-023 · doi:10.1139/y2012-023
- 10.1126/science.1101637 · DOI · Paper page
- 10.1021/jm501562a · DOI · Paper page
- 10.3109/10428194.2012.660631 · DOI
- 10.1182/blood-2004-09-3534 · DOI
- 10.1371/journal.pone.0053673 · DOI
- 10.1158/0008-5472.CAN-10-3870 · DOI
- F. Carles S. Bourg C. Meyer P. Bonnet PKIDB: A curated annotated and updated database of protein kinase inhibitors in clinical trials. Molecules 23 908 (2018). · DOI · Paper page
- 10.1007/s10555-012-9375-7 · DOI · Paper page
- 10.1186/1741-7007-7-50 · DOI · Paper page
- 10.1038/24904 · DOI
- 10.1109/TIT.2020.2996543 · DOI
- 10.1093/bioinformatics/btaa1005 · DOI · Paper page
- B. Ramsundar “Molecular machine learning with DeepChem ” PhD thesis (Stanford University 2018).
- 10.1038/s41598-021-03431-4 · doi:10.1038/s41598-021-03431-4
- I. Gulrajani D. Lopez-Paz In search of lost domain generalization. arXiv [Preprint] (2020). http://arxiv.org/abs/2007.01434 (Accessed 7 December 2022). · arXiv
- 10.1371/journal.pbio.1001244 · DOI · Paper page
- 10.1021/acs.jpcb.2c04525 · DOI
- B. L. Hie K. K. Yang P. S. Kim Evolutionary velocity with protein language models. bioRxiv [Preprint] (2021). https://doi.org/10.1101/2021.06.07.447389 (Accessed 7 December 2022). · DOI
- C. Hsu H. Nisonoff C. Fannjiang J. Listgarten Combining evolutionary and assay-labelled data for protein fitness prediction. bioRxiv [Preprint] (2021). https://doi.org/10.1101/2021.03.28.437402. · DOI
- W. Jin R. Barzilay T. Jaakkola “Junction tree variational autoencoder for molecular graph generation” in International Conference on Machine Learning (PMLR 2018) pp. 2323–2332.
- W. Jin R. Barzilay T. Jaakkola “Hierarchical generation of molecular graphs using structural motifs” in International Conference on Machine Learning (PMLR 2020) pp. 4839–4848.
- 10.1016/j.ejmech.2021.113705 · DOI · Paper page
- 10.1093/nar/gkl999 · DOI · Paper page
- 10.1021/acschembio.7b00996 · DOI · Paper page
- 10.1038/s41589-018-0154-9 · DOI
- 10.1021/acscentsci.9b00835 · DOI · Paper page
- 10.1038/nchembio.1387 · DOI
- 10.1021/ci100050t · DOI · Paper page
- 10.1021/acs.jcim.7b00616 · DOI · Paper page
- H. Wang Chemical-reaction-aware molecule representation learning. arXiv [Preprint] (2021). http://arxiv.org/abs/2109.09888 (Accessed 7 December 2022). · arXiv
- X. Glorot Y. Bengio “Understanding the difficulty of training deep feedforward neural networks” in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (JMLR Workshop and Conference Proceedings 2010) pp. 249–256.
Cited by
Bilingual language model for protein sequence and structure
Heinzinger, M., Weissenow, K., Sanchez, J. G., Henkel, A., Mirdita, M., Steinegger, M., & Rost, B. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae150 · Cited by 311
Democratizing protein language models with parameter-efficient fine-tuning
Sledzieski, S., Kshirsagar, M., Baek, M., Dodhia, R., Lavista Ferres, J., & Berger, B. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2405840121 · Cited by 86
Self-citationArtificial intelligence for drug discovery and development in Alzheimer's disease
Qiu, Y., & Cheng, F. (2024) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2024.102776 · Cited by 83
(2024) · doi:10.1038/s42256-024-00855-1 · Cited by 57
Learning the language of protein-protein interactions
(2026) · doi:10.1038/s41467-025-67971-3 · Cited by 26
Metadata
- Authors
- Singh, R., Sledzieski, S., Bryson, B., Cowen, L., & Berger, B.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2023-06-08
- Volume
- 120
- Issue
- 24
- Pages
- e2220778120-e2220778120
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W4379932151
- Cited by
- 196
- References
- 28
- FWCI
- 26.45
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Funders
- HHS | National Institutes of Health (R35GM141861)
- National Science Foundation (CCF-19345533, 2141064)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.