Scientific Large Language Models: A Survey on Biological & Chemical Domains
Open accessThe DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index.
Abstract
Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelligence. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. This growing interest has led to the advent of scientific LLMs, a novel subclass specifically engineered for facilitating scientific discovery. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration. However, a systematic and up-to-date survey introducing them is currently lacking. In this article, we endeavor to methodically delineate the concept of “scientific language,” whilst providing a thorough review of the latest advancements in scientific LLMs. Given the expansive realm of scientific disciplines, our analysis adopts a focused lens, concentrating on the biological and chemical domains. This includes an in-depth examination of LLMs for textual knowledge, small molecules, macromolecular proteins, genomic sequences, and their combinations, analyzing them in terms of model architectures, capabilities, datasets, and evaluation. Finally, we critically examine the prevailing challenges and point out promising research directions along with the advances of LLMs. By offering a comprehensive overview of technical developments in this field, this survey aspires to be an invaluable resource for researchers navigating the intricate landscape of scientific LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816
DrugBank 5.0: a major update to the DrugBank database for 2018
Wishart, D. S., Feunang, Y. D., Guo, A. C., Lo, E. J., Marcu, A., Grant, J. R., Sajed, T., Johnson, D., Li, C., Sayeeda, Z., et al. (2018) · Nucleic Acids Research · doi:10.1093/nar/gkx1037 · Cited by 8,404
A revision of bloom's taxonomy: an overview
Krathwohl, D. R. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4104_2 · Cited by 8,132
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkab1061 · Cited by 7,850
Weininger, D. (1988) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00057a005 · Cited by 7,436 · cited 1×
“For example, the character “C” in English represents the amino acid Cysteine in protein languages [71], while in the SMILES language system, it denotes a Carbon atom [231].”
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
UniProt: the Universal Protein Knowledgebase in 2023
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., Bye-A-Jee, H., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkac1052 · Cited by 7,210
Self-citationUniProt: the universal protein knowledgebase in 2021
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Agivetova, R., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1100 · Cited by 6,372
Self-citationModeling Relational Data with Graph Convolutional Networks
Schlichtkrull, M., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_38 · Cited by 5,570
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 1×
“Prot-Trans [58] trained several auto-encoder models (includes BERT [104], Albert [110], Electra [42]) on a vast of sequence data.”
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
PubChem in 2021: new data content and improved web interfaces
Kim, S., Chen, J., Cheng, T., Gindulyte, A., He, J., He, S., Li, Q., Shoemaker, B. A., Thiessen, P. A., Yu, B., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa971 · Cited by 3,953
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280
ZINC 15 – Ligand Discovery for Everyone
Sterling, T., & Irwin, J. J. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00559 · Cited by 3,224
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218 · cited 3×
“SciBERT [16] 2019.09 117M BERT Semantic Scholar ScholarBERT [85] 2023.05 340M, 770M BERT Wiki, Books, etc. DARWIN-Base [243] 2023.08 7B LLaMA SciQ, Web of Science SciGLM [264] 2024.01 6B, 32B ChatGLM3 SciInstruct Uni-SMART [29] 2024.03 7B — Patents, news, literature, etc. × SciDFM[204] 2024.09…”
InfluentialMoleculeNet: a benchmark for molecular machine learning
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., & Pande, V. (2018) · Chemical Science · doi:10.1039/c7sc02664a · Cited by 3,107 · cited 1×
“We also curate a collection of benchmarks for Mol-LLMs, including MoleculeNet [237], MARCEL [289], GuacaMol [26], MOSES [170], and ADMETlab 2.0 [244] (see Table 4), which can be effectively utilized for training and evaluating the performance in various molecular tasks.”
InfluentialBERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Self-citationDirectory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking.
Mysinger, M. M., Carchia, M., Irwin, J. J., & Shoichet, B. K. (2012) · Journal of Medicinal Chemistry · doi:10.1021/jm300687e · Cited by 2,412 · cited 1×
“Protein-molecule interaction datasets are an important resource in the field of bioinformatics and drug discovery, providing detailed information on interactions be-tween proteins and small molecule ligands (e.g., DUD-E [157], BioLiP [253], and BindingDB [72]).”
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · cited 2×
“BioBert [111], BioMegatron [193], PubMedBERT [73], BioM-BERT [7], and LinkBERT [255] initially trained on broad corpora like Wikipedia and text-books and then fine-tuned on specific biological NLP tasks, showcase their significant improvements in biological terminology understanding.”
InfluentialA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationPollard, T. J., Johnson, A. E. W., Raffa, J. D., Celi, L. A., Mark, R. G., & Badawi, O. (2018) · Scientific Data · doi:10.1038/sdata.2018.178 · Cited by 1,993
UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches
Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B., Wu, C. H., & the UniProt Consortium (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu739 · Cited by 1,928
Self-citationJudging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 1×
“These similarity metrics include BLEU (Bilingual Evaluation Understudy) [166], ROUGE (Recall-oriented Understudy for Gisting Evaluation) [122], BERT-Score [271], and LLM-Score [281].”
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.10555 · Cited by 1,844
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
UniRef: comprehensive and non-redundant UniProt reference clusters
Suzek, B. E., Huang, H., McGarvey, P., Mazumder, R., & Wu, C. H. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm098 · Cited by 1,613
Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17
Ruddigkeit, L., van Deursen, R., Blum, L. C., & Reymond, J.-L. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci300415d · Cited by 1,505
Effective gene expression prediction from sequence by integrating long-range interactions
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y., Jumper, J., Kohli, P., & Kelley, D. R. (2021) · Nature Methods · doi:10.1038/s41592-021-01252-x · Cited by 1,326 · cited 1×
“Enformer [10] and LOGO [254] employ a blend of convolutional downsampling to handle longer DNA sequences efficiently.”
Protein Data Bank: the single global archive for 3D macromolecular structure data
wwPDB consortium, Burley, S. K., Berman, H. M., Bhikadiya, C., Bi, C., Chen, L., Costanzo, L. D., Christie, C., Duarte, J. M., Dutta, S., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky949 · Cited by 1,163
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
ProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009
Database resources of the National Center for Biotechnology Information
Wheeler, D. L., Barrett, T., Benson, D. A., Bryant, S. H., Canese, K., Chetvernin, V., Church, D. M., DiCuccio, M., Edgar, R., Federhen, S., et al. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm1000 · Cited by 1,007
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Self-citationThe PDBbind database: methodologies and updates
Wang, R., Fang, X., Lu, Y., Yang, C.-Y., & Wang, S. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm048957q · Cited by 933
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908
GuacaMol: Benchmarking Models for de Novo Molecular Design
Brown, N., Fiscato, M., Segler, M. H. S., & Vaucher, A. C. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00839 · Cited by 895
Clustering huge protein sequence sets in linear time
Steinegger, M., & Söding, J. (2018) · Nature Communications · doi:10.1038/s41467-018-04964-5 · Cited by 875 · cited 2×
“The former, which often lacks labels, is commonly employed for self-supervised pre-training, such as UniProtKB (Swiss-Prot and TrEMBL) [22, 155], Pfam [63], and BFD [99, 196, 197].”
InfluentialMolecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models
Polykovskiy, D., Zhebrak, A., Sanchez-Lengeling, B., Golovanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Artamonov, A., Aladinskiy, V., Veselov, M., et al. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.565644 · Cited by 849 · cited 3×
“We also curate a collection of benchmarks for Mol-LLMs, including MoleculeNet [237], MARCEL [289], GuacaMol [26], MOSES [170], and ADMETlab 2.0 [244] (see Table 4), which can be effectively utilized for training and evaluating the performance in various molecular tasks.”
InfluentialSelf-citationProtGPT2 is a deep unsupervised language model for protein design
Ferruz, N., Schmidt, S., & Höcker, B. (2022) · Nature Communications · doi:10.1038/s41467-022-32007-7 · Cited by 795
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
BRENDA, the ELIXIR core data resource in 2021: new developments and updates
Chang, A., Jeske, L., Ulbrich, S., Hofmann, J., Koblitz, J., Schomburg, I., Neumann-Schaal, M., Jahn, D., & Schomburg, D. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1025 · Cited by 789
Nucleotide Transformer: building and evaluating robust foundation models for human genomics
Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski, A. H., Oteri, F., Dallago, C., Trop, E., de Almeida, B. P., Sirelkhatim, H., et al. (2025) · Nature Methods · doi:10.1038/s41592-024-02523-z · Cited by 772
Evaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719 · cited 2×
“The latter, with labeled data, is utilized for supervised fine-tuning or model evaluation, including EC [153], GO [9], ProteinGym [161], FLIP [49], PEER [248], and TAPE [175].”
InfluentialJin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718
Massive mining of publicly available RNA-seq data from human and mouse
Lachmann, A., Torre, D., Keenan, A. B., Jagodnik, K. M., Lee, H. J., Wang, L., Silverstein, M. C., & Ma’ayan, A. (2018) · Nature Communications · doi:10.1038/s41467-018-03751-6 · Cited by 701
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685
The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics
Dalla-Torre, H., Gonzalez, L., Mendoza Revilla, J., Lopez Carranza, N., Henryk Grywaczewski, A., Oteri, F., Dallago, C., Trop, E., De Almeida, B. P., Sirelkhatim, H., et al. (2023) · doi:10.1101/2023.01.11.523679 · Cited by 680
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644 · cited 1×
“In contrast to those primarily centered on nat-ural languages [252, 279], our emphasis leans more toward scientific languages.”
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Yin, B., & Hu, X. (2023) · arXiv · doi:10.48550/arxiv.2304.13712 · Cited by 644
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., & Yao, J. (2021) · doi:10.1101/2021.12.05.471261 · Cited by 617
Galactica: A Large Language Model for Science
Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., Poulton, A., Kerkez, V., & Stojnic, R. (2022) · arXiv · doi:10.48550/arxiv.2211.09085 · Cited by 605
Large-scale chemical language representations capture molecular structure and properties
Ross, J., Belgodere, B., Chenthamarakshan, V., Padhi, I., Mroueh, Y., & Das, P. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00580-7 · Cited by 590
Sequence modeling and design from molecular to genome scale with Evo
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. (2024) · Science · doi:10.1126/science.ado9336 · Cited by 580
Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.14070 · Cited by 512
Self-citationLearning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · cited 1×
“Unlike surveys concentrating solely on molecule [53, 239], protein [20, 87, 213, 217], or genome data [44], we aim to provide a comprehensive view of language models for chemical and biological research.”
ProGen2: Exploring the boundaries of protein language models
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.002 · Cited by 480
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462
S2ORC: The Semantic Scholar Open Research Corpus
Lo, K., Wang, L. L., Neumann, M., Kinney, R., & Weld, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.447 · Cited by 457
Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold
Steinegger, M., Mirdita, M., & Söding, J. (2019) · Nature Methods · doi:10.1038/s41592-019-0437-4 · Cited by 445 · cited 2×
“The former, which often lacks labels, is commonly employed for self-supervised pre-training, such as UniProtKB (Swiss-Prot and TrEMBL) [22, 155], Pfam [63], and BFD [99, 196, 197].”
InfluentialTalking about Large Language Models
Shanahan, M. (2024) · Communications of the ACM · doi:10.1145/3624724 · Cited by 381
Towards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338
PMC-LLaMA: toward building open-source language models for medicine
Wu, C., Lin, W., Zhang, X., Zhang, Y., Xie, W., & Wang, Y. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae045 · Cited by 300
Self-citationMeasuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.149 · Cited by 211
Self-citationAGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211
Self-citation(2023) · doi:10.1101/2023.12.07.570727 · Cited by 207
DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome
Zhou, Z., Ji, Y., Li, W., Dutta, P., Davuluri, R., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2306.15006 · Cited by 179
Improvements and impacts of GRCh38 human reference on high throughput sequencing data analysis
Guo, Y., Dai, Y., Yu, H., Zhao, S., Samuels, D. C., & Shyr, Y. (2017) · Genomics · doi:10.1016/j.ygeno.2017.01.005 · Cited by 172 · cited 1×
“It treats each Mammalian-Gene [199] 1999.10 — Mammalian Genomic Data GRCh38 [77] 2013.12 3.2G nucleotides Genome Reference, Human, Annotations 690 ChIP-seq [261] 2016.06 — DNA-binding Protein Data DeepSEA [287] 2017.04 — Chromatin Effects 1000 Genomes Project [45] 2017.10 20.5T nucleotides Human…”
InfluentialHyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Nguyen, E., Poli, M., Faizi, M., Thomas, A., Birch-Sykes, C., Wornow, M., Patel, A., Rabideau, C., Massaroli, S., Bengio, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.15794 · Cited by 168
DNA language models are powerful predictors of genome-wide variant effects
Benegas, G., Batra, S. S., & Song, Y. S. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311219120 · Cited by 157
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2022) · arXiv · doi:10.48550/arxiv.2203.14371 · Cited by 157
Baichuan 2: Open Large-scale Language Models
Yang, A., Xiao, B., Wang, B., Zhang, B., Bian, C., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.10305 · Cited by 125
Self-citationMEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Chen, Z., Cano, A. H., Romanou, A., Bonnet, A., Matoba, K., Salvi, F., Pagliardini, M., Fan, S., Köpf, A., Mohtashami, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.16079 · Cited by 122
ChemCrow: Augmenting large-language models with chemistry tools
Bran, A. M., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2023) · arXiv · doi:10.48550/arxiv.2304.05376 · Cited by 122
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2023) · The International Journal of High Performance Computing Applications · doi:10.1177/10943420231201154 · Cited by 120
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2022) · doi:10.1101/2022.10.10.511571 · Cited by 120 · cited 1×
“GenSLMs [292] leverage genome-scale language models comprising multiple layers of attention-based decoders to elucidate the evolutionary dynamics of SARS-CoV-2.”
GPN-MSA: an alignment-based DNA language model for genome-wide variant effect prediction
Benegas, G., Albors, C., Aw, A. J., Ye, C., & Song, Y. S. (2023) · doi:10.1101/2023.10.10.561776 · Cited by 118 · cited 1×
“…[94], DNABERT-2 [288], iEnhancer-BERT [140], scBERT [251], BioSeq-BLM [114], PLPMpro [120], GPN [18], SA DNA-LM [68], RNA-MSM [275], miProBERT [224], TFBert [141], MRM-BERT [274], GPN-MSA [17], iDNA-ABF [96], and GET [67] are noteworthy attempts employing the Transformer encoder within genomics.”
DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task
Xiong, H., Wang, S., Zhu, Y., Zhao, Z., Liu, Y., Huang, L., Wang, Q., & Shen, D. (2023) · arXiv · doi:10.48550/arxiv.2304.01097 · Cited by 81
Developing ChemDFM as a large language foundation model for chemistry
Zhao, Z., Ma, D., Chen, L., Sun, L., Li, Z., Xia, Y., Chen, B., Xu, H., Zhu, Z., Zhu, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.14818 · Cited by 48
Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding
Toma, A., Lawler, P. R., Ba, J., Krishnan, R. G., Rubin, B. B., & Wang, B. (2023) · arXiv · doi:10.48550/arxiv.2305.12031 · Cited by 36
Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation
Gu, Z., Zhu, X., Ye, H., Zhang, L., Wang, J., Zhu, Y., Jiang, S., Xiong, Z., Li, Z., Wu, W., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29767 · Cited by 28
Self-citationMedDialog: Two Large-scale Medical Dialogue Datasets
He, X., Chen, S., Ju, Z., Dong, X., Fang, H., Wang, S., Yang, Y., Zeng, J., Zhang, R., Zhang, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.03329 · Cited by 12
Schmidinger, N., Schneckenreiter, L., Seidl, P., Schimunek, J., Hoedt, P.-J., Brandstetter, J., Mayr, A., Luukkonen, S., Hochreiter, S., & Klambauer, G. (2024) · arXiv · doi:10.48550/arxiv.2411.04165 · Cited by 8
Research works printed but not linked: 221
- 10.1021/acs.jcim.2c00715 · DOI
- Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, and Michalis Vazirgiannis. 2024. Prot2text: Multimodal protein’s function generation with gnns and transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 10757–10765.
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat et al. 2023. Gpt-4 technical report. Retrieved from https://arxiv.org/abs/2303.08774 · arXiv
- Sanjar Adilov. 2021. Generative pre-training from molecules. Retrieved from https://chemrxiv.org/engage/chemrxiv/article-details/6142f60742198e8c31782e9e · DOI
- Walid Ahmad, Elana Simon, Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. 2022. Chemberta-2: Towards chemical foundation models. Retrieved from https://arxiv.org/abs/2209.01712 · arXiv
- Microsoft Research AI4Science and Microsoft Azure Quantum. 2023. The impact of large language models on scientific discovery: A preliminary study using gpt-4. Retrieved from https://arxiv.org/abs/2311.07361 · arXiv
- Sultan Alrowili and K. Vijay-Shanker. 2021. BioM-transformers: Building large biomedical language models with BERT, ALBERT, and ELECTRA. In Proceedings of the 20th Workshop on Biomedical Language Processing. 221–227. · DOI
- Weizhi An, Yuzhi Guo, Yatao Bian, Hehuan Ma, Jinyu Yang, Chunyuan Li, and Junzhou Huang. 2022. MoDNA: Motif-oriented pre-training for DNA language model. In Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. 1–5.
- 10.1038/75556 · DOI · Paper page
- 10.1038/s41597-022-01288-4 · DOI · Paper page
- 10.1021/acs.jcim.1c00600 · DOI · Paper page
- 10.1093/nar/28.1.45 · DOI · Paper page
- Suryanarayanan Balaji, Rishikesh Magar, Yayati Jadhav et al. 2023. GPT-MolBERTa: GPT molecular features language model for molecular property prediction. Retrieved from https://arxiv.org/abs/2310.03030 · arXiv
- Dominique Beaini, Shenyang Huang, Joao Alex Cunha, Gabriela Moisescu-Pareja, Oleksandr Dymov, Samuel Maddrell-Mander, Callum McLean, Frederik Wenkel, Luis Müller, Jama Hussein Mohamud et al. 2023. Towards foundational models for molecular learning on large-scale multi-task datasets. Retrieved from https://arxiv.org/abs/2310.04292 · arXiv
- Gonzalo Benegas Sanjit Singh Batra and Yun S. Song. 2022. DNA language models are powerful zero-shot predictors of genome-wide variant effects. Retrieved from 10.1101/2022.08.22.504706v1 · doi:10.1101/2022.08.22.504706v1
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 26 (2013).
- Emmanuel Boutet, Damien Lieberherr, Michael Tognolli, Michel Schneider, Parit Bansal, Alan J. Bridge, Sylvain Poux, Lydie Bougueleret, and Ioannis Xenarios. 2016. UniProtKB/Swiss-Prot, the manually annotated section of the UniProt KnowledgeBase: How to use the entry view. Plant Bioinform.: Methods Protocols (2016), 23–54. · DOI · Paper page
- Andres M. Bran and Philippe Schwaller. 2024. Transformers and large language models for chemistry and drug discovery. In Drug Development Supported by Informatics. Springer, 143–163. · DOI
- Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. Retrieved from https://arxiv.org/abs/2303.12712 · arXiv
- Hengxing Cai, Xiaochen Cai, Shuwen Yang, Jiankun Wang, Lin Yao, Zhifeng Gao, Junhan Chang, Sihang Li, Mingjun Xu, Changxin Wang et al. 2024. Uni-SMART: Universal science multimodal analysis and research transformer. Retrieved from https://arXiv:2403.10301
- Bo Chen, Xingyi Cheng, Pan Li, Yangli-ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Zeng et al. 2024. xTrimoPGLM: Unified 100B-scale pre-trained transformer for deciphering the language of protein. Retrieved from https://arxiv.org/abs/2401.06199 · arXiv
- Dong Chen, Kaifu Gao, Duc Duy Nguyen, Xin Chen, Yi Jiang, Guo-Wei Wei, and Feng Pan. 2021. Algebraic graph-assisted bidirectional transformers for molecular property prediction. Nature Commun. 12, 1 (2021), 3521. · DOI · Paper page
- Ken Chen Yue Zhou Maolin Ding Yu Wang Zhixiang Ren and Yuedong Yang. 2023. Self-supervised learning on millions of pre-mRNA sequences improves sequence-based RNA splicing prediction. Retrieved from 10.1101/2023.01.31.526427v1 · doi:10.1101/2023.01.31.526427v1
- Qiyuan Chen and Cheng Deng. 2023. Bioinfo-Bench: A simple benchmark framework for LLM bioinformatics skills evaluation. Retrieved from 10.1101/2023.10.18.563023v1 · doi:10.1101/2023.10.18.563023v1
- Yangyang Chen, Zixu Wang, Lei Wang, Jianmin Wang, Pengyong Li, Dongsheng Cao, Xiangxiang Zeng, Xiucai Ye, and Tetsuya Sakurai. 2023. Deep generative model for drug design from protein target sequence. J. Cheminform. 15, 1 (2023), 38. · DOI
- Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. Retrieved 14 April 2023 from https://vicuna.lmsys.org
- Gayane Chilingaryan, Hovhannes Tamoyan, Ani Tevosyan, Nelly Babayan, Lusine Khondkaryan, Karen Hambardzumyan, Zaven Navoyan, Hrant Khachatrian, and Armen Aghajanyan. 2022. Bartsmiles: Generative masked language models for molecular representations. Retrieved from https://arxiv.org/abs/2211.16349 · arXiv
- Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. 2020. ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction. Retrieved from https://arxiv.org/abs/2010.09885 · arXiv
- Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Matteo Manica. 2023. Unifying molecular and textual representations via multi-task language modelling. In Proceedings of the International Conference on Machine Learning. PMLR, 6140–6157.
- Simon K. S. Chu and Kathy Y. Wei. 2023. Generative antibody design for complementary chain pairing sequences through encoder-decoder language model. Retrieved from https://arxiv.org/abs/2301.02748 · arXiv
- Micaela E. Consens, Cameron Dufault, Michael Wainberg, Duncan Forster, Mehran Karimzadeh, Hani Goodarzi, Fabian J. Theis, Alan Moses, and Bo Wang. 2023. To transformers and beyond: Large language models for the genome. Retrieved from https://arxiv.org/abs/2311.07621 · arXiv
- 10.1038/nature15393 · DOI · Paper page
- ENCODE Project Consortium Overall coordination (data analysis coordination) Dunham Ian 2 Kundaje Anshul 3 81 82 82, Writing group Bernstein Bradley E. 7 34 Birney Ewan Dunham Ian Green Eric D. 35 Gunter Chris 15 Snyder Michael 13 et al. 2012. An integrated encyclopedia of DNA elements in the human genome. Nature 489, 7414 (2012), 57–74. · DOI · Paper page
- Christian Dallago, Jody Mou, Kadina E. Johnston, Bruce J. Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K. Yang. 2021. FLIP: Benchmark tasks in fitness landscape inference for proteins. Retrieved from https://www.biorxiv.org/content/2021.11.09.467890
- Lucas Paulo de Lima Camillo Raghav Sehgal Jenel Armstrong Albert Tzongyang Higgins-Chen Steve Horvath and Bo Wang. 2024. CpGPT: A foundation model for DNA methylation. Retrieved from 10.1101/2024.10.24.619766v1 · doi:10.1101/2024.10.24.619766v1
- Yifan Deng, Spencer S. Ericksen, and Anthony Gitter. 2024. Chemical language model linker: Blending text and molecules with modular adapters. Retrieved from https://arXiv:2410.20182
- René Dreos, Giovanna Ambrosini, Rouayda Cavin Périer, and Philipp Bucher. 2013. EPD and EPDnew, high-quality promoter resources in the next-generation sequencing era. Nucl. Acids Res. 41, D1 (2013), D157–D164.
- Yuanqi Du, Tianfan Fu, Jimeng Sun, and Shengchao Liu. 2022. Molgensurvey: A systematic survey in machine learning models for molecule design. Retrieved from https://arxiv.org/abs/2203.14500 · arXiv
- Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2021. Glm: General language model pretraining with autoregressive blank infilling. Retrieved from https://arxiv.org/abs/2103.10360 · arXiv
- Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. 2022. Translation between molecules and natural language. Retrieved from https://arxiv.org/abs/2204.11817 · arXiv
- Carl Edwards, Ziqing Lu, Ehsan Hajiramezanali, Tommaso Biancalani, Heng Ji, and Gabriele Scalia. 2024. MolCap-Arena: A comprehensive captioning benchmark on language-enhanced molecular property prediction. Retrieved from https://arXiv:2411.00737
- Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. 2023. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. Retrieved from https://arxiv.org/abs/2306.08018 · arXiv
- Yin Fang, Ningyu Zhang, Zhuo Chen, Lingbing Guo, Xiaohui Fan, and Huajun Chen. 2023. Domain-agnostic molecular generation with self-feedback. Retrieved from https://arxiv.org/abs/2301.11259 · arXiv
- Kehua Feng, Keyan Ding, Weijie Wang, Xiang Zhuang, Zeyuan Wang, Ming Qin, Yu Zhao, Jianhua Yao, Qiang Zhang, and Huajun Chen. 2024. SciKnowEval: Evaluating multi-level scientific knowledge of large language models. Retrieved from https://arxiv.org/abs/2406.09098 · arXiv
- Robert D. Finn, Jaina Mistry, Benjamin Schuster-Böckler, Sam Griffiths-Jones, Volker Hollich, Timo Lassmann, Simon Moxon, Mhairi Marshall, Ajay Khanna, Richard Durbin et al. 2006. Pfam: Clans, web tools and services. Nucl. Acids Res. 34, suppl_1 (2006), D247–D251.
- Veniamin Fishman, Yuri Kuratov, Maxim Petrov, Aleksei Shmelev, Denis Shepelin, Nikolay Chekanov, Olga Kardymon, and Mikhail Burtsev. 2023. GENA-LM: A family of open-source foundational models for long DNA sequences. Retrieved from https://www.biorxiv.org/content/2023.06.12.544594
- Mary Forehand. 2010. Bloom’s taxonomy. Emerg. Perspect. Learn. Teach. Technol. 41, 4 (2010), 47–56.
- Oscar Franzén, Li-Ming Gan, and Johan L. M. Björkegren. 2019. PanglaoDB: A web server for exploration of mouse and human single-cell RNA sequencing data. Database 2019 (2019), baz046.
- Xi Fu, Shentong Mo, Alejandro Buendia, Anouchka P. Laurent, Anqi Shao, Maria del Mar Alvarez-Torres, Tianji Yu, Jimin Tan, Jiayu Su, Romella Sagatelian et al. 2025. A foundation model of transcription across human cell types. Nature (2025), 1–9.
- Dennis Gankin, Alexander Karollus, Martin Grosshauser, Kristian Klemon, Johannes Hingerl, and Julien Gagneur. 2023. Species-aware DNA language modeling. Retrieved from https://www.biorxiv.org/content/2023.01.26.525670
- Bowen Gao, Bo Qiang, Haichuan Tan, Yinjun Jia, Minsi Ren, Minsi Lu, Jingjing Liu, Wei-Ying Ma, and Yanyan Lan. 2024. DrugCLIP: Contrasive protein-molecule representation learning for virtual screening. Adv. Neural Inf. Process. Syst. 36 (2024).
- Zijing Gao, Qiao Liu, Wanwen Zeng, Wing H. Wong, and Rui Jiang. 2023. EpiGePT: A pretrained transformer model for epigenomics. Retrieved from https://bioRxiv:2023.07.15.549134
- H. Bielka, N. Sharon, and E. W. Australia. 1984. Nomenclature and symbolism for amino acids and peptides. Pure Appl. Chem. 56 (1984), 595–624. · DOI
- Michael K. Gilson, Tiqing Liu, Michael Baitaluk, George Nicola, Linda Hwang, and Jenny Chong. 2016. BindingDB in 2015: A public database for medicinal chemistry, computational chemistry and systems pharmacology. Nucl. Acids Res. 44, D1 (2016), D1045–D1053.
- Jiang Guo, A. Santiago Ibanez-Lopez, Hanyu Gao, Victor Quach, Connor W. Coley, Klavs F. Jensen, and Regina Barzilay. 2021. Automated chemical reaction extraction from scientific literature. J. Chem. Inf. Model. 62, 9 (2021), 2035–2045.
- Xiao-Yu Guo, Yi-Fan Li, Yuan Liu, Xiaoyong Pan, and Hong-Bin Shen. 2024. ProtDAT: A unified framework for protein sequence design from any protein text description. Retrieved from https://arXiv:2412.04069
- Tanishq Gupta, Mohd Zaki, N. M. Anoop Krishnan, and Mausam. 2022. MatSciBERT: A materials domain language model for text mining and information extraction. npj Comput. Mater. 8, 1 (2022), 102. · DOI · Paper page
- Liang He, Shizhuo Zhang, Lijun Wu, Huanhuan Xia, Fusong Ju, He Zhang, Siyuan Liu, Yingce Xia, Jianwei Zhu, Pan Deng et al. 2021. Pre-training co-evolutionary protein representation via a pairwise masked language model. Retrieved from https://arxiv.org/abs/2110.15527 · arXiv
- Michael Heinzinger, Konstantin Weissenow, Joaquin Gomez Sanchez, Adrian Henkel, Martin Steinegger, and Burkhard Rost. 2023. ProstT5: Bilingual language model for protein sequence and structure. Retrieved from https://www.biorxiv.org/content/2023.07.23.550085
- Daniel Hesslow, Niccoló Zanichelli, Pascal Notin, Iacopo Poli, and Debora Marks. 2022. Rita: A study on scaling up generative protein sequence models. Retrieved from https://arxiv.org/abs/2205.05789 · arXiv
- David Hiscock and Chris Upton. 2000. Viral genome DataBase: Storing and analyzing genes and proteins from complete viral genomes. Bioinformatics 16, 5 (2000), 484–485. · doi:10.1093/bioinformatics/16.5.484
- Shion Honda, Shoi Shi, and Hiroki R. Ueda. 2019. Smiles transformer: Pre-trained molecular fingerprint for low data drug discovery. Retrieved from https://arxiv.org/abs/1911.04738 · arXiv
- Zhi Hong, Aswathy Ajith, Gregory Pauloski, Eamon Duede, Kyle Chard, and Ian Foster. 2022. The diminishing returns of masked language models to science. Retrieved from https://arxiv.org/abs/2205.11342 · arXiv
- Marc Horlacher, Giulia Cantini, Julian Hesse, Patrick Schinke, Nicolas Goedert, Shubhankar Londhe, Lambert Moyon, and Annalisa Marsico. 2023. A systematic benchmark of machine learning methods for protein–RNA interaction prediction. Brief. Bioinform. 24, 5 (2023), bbad307. · DOI
- Bozhen Hu, Jun Xia, Jiangbin Zheng, Cheng Tan, Yufei Huang, Yongjie Xu, and Stan Z. Li. 2022. Protein language models and structure prediction: Connection and progression. Retrieved from https://arxiv.org/abs/2211.16742 · arXiv
- Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. 2021. Ogb-lsc: A large-scale challenge for machine learning on graphs. Retrieved from https://arxiv.org/abs/2103.09430 · arXiv
- Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Adv. Neural Inf. Process. Syst. 33 (2020), 22118–22133.
- Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu et al. 2024. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. Adv. Neural Inf. Process. Syst. 36 (2024).
- 10.1021/ci3001277 · DOI · Paper page
- 10.1021/acs.jcim.0c00675 · DOI · Paper page
- 10.1088/2632-2153/ac3ffb · DOI · Paper page
- 10.1093/bioinformatics/btab083 · DOI · Paper page
- Minghui Jiang, Ying Xu, and Binhai Zhu. 2008. Protein structure–structure alignment with discrete Fréchet distance. J. Bioinform. Comput. Biol. 6, 01 (2008), 51–64. · doi:10.1142/s0219720008003278
- Junru Jin, Yingying Yu, Ruheng Wang, Xin Zeng, Chao Pang, Yi Jiang, Zhongshen Li, Yutong Dai, Ran Su, Quan Zou et al. 2022. iDNA-ABF: Multi-scale deep biological language learning model for the interpretable prediction of DNA methylations. Genome Biol. 23, 1 (2022), 1–23.
- Mingyu Jin Haochen Xue Zhenting Wang Boming Kang Ruosong Ye Kaixiong Zhou Mengnan Du and Yongfeng Zhang. 2024. ProLLM: Protein chain-of-thoughts enhanced LLM for protein-protein interaction prediction. Retrieved from https://arxiv.org/abs/2405.06649 · arXiv
- Yoav Kan-Tor, Michael Morris Danziger, Eden Zohar, Matan Ninio, and Yishai Shimoni. 2024. Does your model understand genes? A benchmark of gene properties for biological and text models. Retrieved from https://arxiv.org/abs/2412.04075 · arXiv
- 10.1093/nar/gkg129 · DOI · Paper page
- Pavel Karpov, Guillaume Godin, and Igor V. Tetko. 2019. A transformer model for retrosynthesis. In Proceedings of the International Conference on Artificial Neural Networks. Springer, 817–830.
- Panagiotis Katsonis and Olivier Lichtarge. 2019. CAGI5: Objective performance assessments of predictions based on the Evolutionary Action equation. Hum. Mutat. 40, 9 (2019), 1436–1454. · DOI
- Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19), Vol. 1. Minneapolis, Minnesota, 2.
- Hyunseung Kim, Jonggeol Na, and Won Bo Lee. 2021. Generative chemical transformer: Neural machine learning of molecular geometric structures from chemical language via attention. J. Chem. Inf. Model. 61, 12 (2021), 5804–5814. · DOI · Paper page
- Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A. Shoemaker, Paul A. Thiessen, Bo Yu et al. 2023. PubChem 2023 update. Nucl. Acids Res. 51, D1 (2023), D1373–D1380.
- 10.1088/2632-2153/aba947 · DOI · Paper page
- Philippe Lamesch, Tanya Z. Berardini, Donghui Li, David Swarbreck, Christopher Wilks, Rajkumar Sasidharan, Robert Muller, Kate Dreher, Debbie L. Alexander, Margarita Garcia-Hernandez et al. 2012. The arabidopsis information resource (TAIR): Improved gene annotation and new tools. Nucl. Acids Res. 40, D1 (2012), D1202–D1210.
- 10.1145/3534678.3539426 · DOI
- Hong-Liang Li, Yi-He Pang, and Bin Liu. 2021. BioSeq-BLM: A platform for analyzing DNA, RNA and protein sequences based on biological language models. Nucl. Acids Res. 49, 22 (2021), e129–e129.
- Juncai Li and Xiaofei Jiang. 2021. Mol-BERT: An effective molecular representation with BERT for molecular property prediction. Wireless Commun. Mobile Comput. 2021 (2021), 1–7.
- Jiatong Li, Junxian Li, Yunqing Liu, Dongzhan Zhou, and Qing Li. 2024. TOMG-Bench: Evaluating LLMs on text-based open molecule generation. Retrieved from https://arXiv:2412.14642
- Jiahao Li, Zhourun Wu, Wenhao Lin, Jiawei Luo, Jun Zhang, Qingcai Chen, and Junjie Chen. 2023. iEnhancer-ELM: Improve enhancer identification by extracting position-related multiscale contextual information based on enhancer language models. Bioinform. Adv. 3, 1 (2023), vbad043. · DOI
- Yuesen Li, Chengyi Gao, Xin Song, Xiangyu Wang, Yungang Xu, and Suxia Han. 2023. Druggpt: A gpt-based strategy for designing potential ligands targeting specific proteins. Retrieved from https://www.biorxiv.org/content/2023.06.29.543848
- 10.1016/j.compbiomed.2023.107260 · DOI
- Youwei Liang, Ruiyi Zhang, Li Zhang, and Pengtao Xie. 2023. DrugChat: Towards enabling ChatGPT-like capabilities on drug molecule graphs. Retrieved from https://arxiv.org/abs/2309.03907 · arXiv
- Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, 74–81.
- Zeming Lin Halil Akin Roshan Rao Brian Hie Zhongkai Zhu Wenting Lu Allan dos Santos Costa Maryam Fazel-Zarandi Tom Sercu Sal Candido et al. 2022. Language models of protein sequences at the scale of evolution enable accurate structure prediction. Retrieved from 10.1101/2022.07.20.500902v1 · doi:10.1101/2022.07.20.500902v1
- 10.1016/j.compbiomed.2024.108073 · DOI · Paper page
- Shengchao Liu, Yanjing Li, Zhuoxinran Li, Anthony Gitter, Yutao Zhu, Jiarui Lu, Zhao Xu, Weili Nie, Arvind Ramanathan, Chaowei Xiao et al. 2023. A text-guided protein design framework. Retrieved from https://arxiv.org/abs/2302.04611 · arXiv
- 10.1038/s42256-023-00759-6 · DOI · Paper page
- Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. 2021. Pre-training molecular graph representation with 3D geometry. Retrieved from https://arxiv.org/abs/2110.07728 · arXiv
- Shengchao Liu, Jiongxiao Wang, Yijin Yang, Chengpeng Wang, Ling Liu, Hongyu Guo, and Chaowei Xiao. 2023. Chatgpt-powered conversational drug editing using retrieval and domain feedback. Retrieved from https://arxiv.org/abs/2305.18090 · arXiv
- Yuyan Liu, Sirui Ding, Sheng Zhou, Wenqi Fan, and Qiaoyu Tan. 2024. MolecularGPT: Open large language model (LLM) for few-shot molecular property prediction. Retrieved from https://arXiv:2406.12950
- 10.1016/j.jmgm.2022.108344 · DOI
- Zhiyuan Liu, An Zhang, Hao Fei, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, and Tat-Seng Chua. 2024. ProtT3: Protein-to-text generation for text-based protein understanding. Retrieved from https://arXiv:2405.12564
- Zequn Liu, Wei Zhang, Yingce Xia, Lijun Wu, Shufang Xie, Tao Qin, Ming Zhang, and Tie-Yan Liu. 2023. MolXPT: Wrapping molecules with text for generative pre-training. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 1606–1616.
- Daniel Mark Lowe. 2012. Extraction of chemical structures and reactions from the literature. Ph.D. Dissertation, University of Cambridge.
- Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Adv. Neural Inf. Process. Syst. 35 (2022), 2507–2521.
- Xingyu Lu, He Cao, Zijing Liu, Shengyuan Bai, Leqing Chen, Yuan Yao, Hai-Tao Zheng, and Yu Li. 2024. MoleculeQA: A dataset to evaluate factual accuracy in molecular comprehension. Retrieved from https://arXiv:2403.08192
- Yen-Chun Lu Ashley Varghese Rahul Nahar Hao Chen Kunming Shao Xiaoping Bao and Can Li. 2024. scChat: A large language model-powered co-pilot for contextualized single-cell RNA sequencing analysis. Retrieved from 10.1101/2024.10.01.616063v1 · doi:10.1101/2024.10.01.616063v1
- Hanyu Luo, Cheng Chen, Wenyu Shan, Pingjian Ding, and Lingyun Luo. 2022. iEnhancer-BERT: A novel transfer learning architecture based on DNA-Language model for identifying enhancers and their strength. In Proceedings of the International Conference on Intelligent Computing. Springer, 153–165.
- Hanyu Luo, Wenyu Shan, Cheng Chen, Pingjian Ding, and Lingyun Luo. 2023. Improving language model of human genome for DNA–protein binding prediction based on task-specific pre-training. Interdisc. Sci.: Comput. Life Sci. 15, 1 (2023), 32–43.
- Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, and Zaiqing Nie. 2023. Molfm: A multimodal molecular foundation model. Retrieved from https://arxiv.org/abs/2307.09484 · arXiv
- Yizhen Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Yushuai Wu, Mu Qiao, and Zaiqing Nie. 2023. Biomedgpt: Open multimodal generative pre-trained transformer for biomedicine. Retrieved from https://arxiv.org/abs/2308.09442 · arXiv
- Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian. 2024. ProLLaMA: A protein large language model for multi-task protein language processing. Retrieved from https://arXiv:2402.16445
- Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, and Lingpeng Kong. 2023. Retrieved sequence augmentation for protein representation learning. Retrieved from https://www.biorxiv.org/content/2023.02.22.529597
- Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R. Eguchi, Po-Ssu Huang, and Richard Socher. 2020. Progen: Language modeling for protein generation. Retrieved from https://arxiv.org/abs/2004.03497 · arXiv
- Aditya Malusare, Harish Kothandaraman, Dipesh Tamboli, Nadia A. Lanman, and Vaneet Aggarwal. 2023. Understanding the natural language of DNA using encoder-decoder foundation models with byte-level precision. Retrieved from https://arxiv.org/abs/2311.02333 · arXiv
- Vipul Mann and Venkat Venkatasubramanian. 2021. Predicting chemical reaction outcomes: A grammar ontology-based transformer framework. AIChE J. 67, 3 (2021), e17190. · DOI
- Jiashun Mao Jianmin Wang Kwang-Hwi Cho and Kyoung Tai No. 2023. iupacGPT: IUPAC-based large-scale molecular pre-trained model for property prediction and molecule generation. Retrieved from https://chemrxiv.org/engage/chemrxiv/article-details/645f49f9a32ceeff2d90c9ae
- Łukasz Maziarka, Tomasz Danel, Sławomir Mucha, Krzysztof Rataj, Jacek Tabor, and Stanisław Jastrzębski. 2020. Molecule attention transformer. Retrieved from https://arxiv.org/abs/2002.08264 · arXiv
- Eyal Mazuz, Guy Shtar, Bracha Shapira, and Lior Rokach. 2023. Molecule generation using transformers and policy gradient reinforcement learning. Sci. Rep. 13, 1 (2023), 8799. · DOI
- Andrew G. McDonald, Sinéad Boyce, and Keith F. Tipton. 2009. ExplorEnz: The primary source of the IUBMB enzyme list. Nucl. Acids Res. 37, suppl_1 (2009), D593–D597.
- Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. 2021. Language models enable zero-shot prediction of the effects of mutations on protein function. Adv. Neural Inf. Process. Syst. 34 (2021), 29287–29303.
- 10.1093/bioinformatics/15.3.219 · doi:10.1093/bioinformatics/15.3.219
- Geraldene Munsamy, Sebastian Lindner, Philipp Lorenz, and Noelia Ferruz. 2022. ZymCTRL: A conditional language model for the controllable generation of artificial enzymes. In Proceedings of the Machine Learning for Structural Biology Workshop. NeurIPS 2022.
- Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena-Hurtado, Aidan N. Gomez, Debora Marks, and Yarin Gal. 2022. Tranception: Protein fitness prediction with autoregressive transformers and inference-time retrieval. In Proceedings of the International Conference on Machine Learning. PMLR, 16990–17017.
- Pascal Notin, Aaron W. Kollasch, Daniel Ritter, Lood van Niekerk, Steffanie Paul, Hansen Spinner, Nathan Rollins, Ada Shaw, Ruben Weitzman, Jonathan Frazeret al.. 2023. ProteinGym: Large-scale benchmarks for protein design and fitness prediction. Retrieved from https://www.biorxiv.org/content/2023.12.07.570727
- Pascal Notin, Ruben Weitzman, Debora S. Marks, and Yarin Gal. 2023. ProteinNPT: Improving protein property prediction and design with non-parametric transformers. Retrieved from https://www.biorxiv.org/content/2023.12.06.570473
- Robert D. Olson, Rida Assaf, Thomas Brettin, Neal Conrad, Clark Cucinell, James J. Davis, Donald M. Dempsey, Allan Dickerman, Emily M. Dietrich, Ronald W. Kenyon et al. 2023. Introducing the bacterial and viral bioinformatics resource center (BV-BRC): A resource combining PATRIC, IRD and ViPR. Nucl. Acids Res. 51, D1 (2023), D678–D689.
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 311–318.
- Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. 2024. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning. Retrieved from https://arXiv:2402.17810
- Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan. 2023. Biot5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations. Retrieved from https://arxiv.org/abs/2310.07276 · arXiv
- Yifan Peng, Qingyu Chen, and Zhiyong Lu. 2020. An empirical study of multi-task learning on BERT for biomedical text mining. Retrieved from https://arxiv.org/abs/2005.02799 · arXiv
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.
- 10.5555/3455716.3455856 · doi:10.5555/3455716.3455856
- Soumya Ram and Tristan Bepler. 2022. Few shot protein generation. Retrieved from https://arxiv.org/abs/2204.01168 · arXiv
- Roshan M. Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. 2021. MSA transformer. In Proceedings of the International Conference on Machine Learning. PMLR, 8844–8856.
- Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. 2020. Self-supervised graph transformer on large-scale molecular data. Adv. Neural Inf. Process. Syst. 33 (2020), 12559–12571.
- Nicole Rusk. 2018. Sequence-based prediction of variants’ effects. Nature Methods 15, 8 (2018), 571–571. · DOI
- 10.1371/journal.pone.0031826 · DOI · Paper page
- 10.1021/acscentsci.9b00576 · DOI · Paper page
- 10.1038/s42256-020-00284-w · DOI · Paper page
- Philipp Seidl, Andreu Vall, Sepp Hochreiter, and Günter Klambauer. 2023. Enhancing activity prediction models in drug discovery with the ability to understand human language. In Proceedings of the International Conference on Machine Learning. PMLR, 30458–30490.
- Bin Shao and Jiawei Yan. 2024. A long-context language model for deciphering and generating bacteriophage genomes. Nature Commun. 15, 1 (2024), 9392. · DOI
- Yiqing Shen, Zan Chen, Michail Mamalakis, Luhan He, Haiyang Xia, Tianbin Li, Yanzhou Su, Junjun He, and Yu Guang Wang. 2024. A fine-tuning dataset and benchmark for large language models for protein understanding. Retrieved from https://arXiv:2406.05540
- Yiqing Shen, Zan Chen, Michail Mamalakis, Yungeng Liu, Tianbin Li, Yanzhou Su, Junjun He, Pietro Liò, and Yu Guang Wang. 2024. Toursynbio: A multi-modal large model and agent framework to bridge text and protein sequences for protein engineering. Retrieved from https://arXiv:2408.15299
- Pranav Shetty, Arunkumar Chitteth Rajan, Chris Kuenneth, Sonakshi Gupta, Lakshmi Prerana Panchumarti, Lauren Holm, Chao Zhang, and Rampi Ramprasad. 2023. A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing. npj Comput. Mater. 9, 1 (2023), 52. · DOI · Paper page
- Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, and Raghav Mani. 2020. BioMegatron: Larger biomedical domain language model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20). 4700–4706.
- Yoel Shoshan, Moshiko Raboh, Michal Ozery-Flato, Vadim Ratner, Alex Golts, Jeffrey K. Weber, Ella Barkan, Simona Rabinovici-Cohen, Sagi Polaczek, Ido Amos et al. 2024. MAMMAL–molecular aligned multi-modal architecture and language. Retrieved from https://arXiv:2410.22367
- Richard W. Shuai Jeffrey A. Ruffolo and Jeffrey J. Gray. 2021. Generative language modeling for antibody design. Retrieved from 10.1101/2021.12.13.472419v2 · doi:10.1101/2021.12.13.472419v2
- Robert L. Strausberg, Elise A. Feingold, Richard D. Klausner, and Francis S. Collins. 1999. The mammalian gene collection. Science 286, 5439 (1999), 455–457. · DOI · Paper page
- Bing Su, Dazhao Du, Zhao Yang, Yujie Zhou, Jiangmeng Li, Anyi Rao, Hao Sun, Zhiwu Lu, and Ji-Rong Wen. 2022. A molecular multimodal foundation model associating molecule graphs with natural language. Retrieved from https://arxiv.org/abs/2209.05481 · arXiv
- Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. 2023. Saprot: Protein language modeling with structure-aware vocabulary. Retrieved from https://www.biorxiv.org/content/2023.10.01.560349
- Jiangming Sun, Nina Jeliazkova, Vladimir Chupakhin, Jose-Felipe Golib-Dzib, Ola Engkvist, Lars Carlsson, Jörg Wegner, Hugo Ceulemans, Ivan Georgiev, Vedrin Jeliazkov et al. 2017. ExCAPE-DB: An integrated large scale dataset facilitating Big Data analysis in chemogenomics. J. Cheminform. 9 (2017), 1–9.
- Liangtai Sun, Yang Han, Zihan Zhao, Da Ma, Zhennan Shen, Baocai Chen, Lu Chen, and Kai Yu. 2024. Scieval: A multi-level large language model evaluation benchmark for scientific research. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19053–19061.
- Liangtai Sun, Danyu Luo, Da Ma, Zihan Zhao, Baocai Chen, Zhennan Shen, Su Zhu, Lu Chen, Xin Chen, and Kai Yu. 2024. SciDFM: A large language model with mixture-of-experts for science. Retrieved from https://arXiv:2409.18412
- 10.1038/s41467-020-19266-y · DOI · Paper page
- Amol Thakkar, Alain C. Vaucher, Andrea Byekwaso, Philippe Schwaller, Alessandra Toniato, and Teodoro Laino. 2023. Unbiasing retrosynthesis language models with disconnection prompts. ACS Central Sci. 9, 7 (2023), 1488–1498. · DOI
- Alessandra Toniato, Alain C. Vaucher, Philippe Schwaller, and Teodoro Laino. 2023. Enhancing diversity in language based models for single-step retrosynthesis. Dig. Discov. 2, 2 (2023), 489–501. · DOI
- Chau Tran, Siddharth Khadkikar, and Aleksey Porollo. 2023. Survey of protein sequence embedding models. Int. J. Mol. Sci. 24, 4 (2023), 3775. · DOI
- 10.3390/ijms241511948 · DOI
- 10.1038/s41467-022-28857-w · DOI
- Gökçe Uludoğan, Elif Ozkirimli, Kutlu O. Ulgen, Nilgün Karalı, and Arzucan Özgür. 2022. Exploiting pretrained biochemical language models for targeted drug design. Bioinformatics 38, Supplement_2 (2022), ii155–ii161.
- 10.1038/s42256-022-00457-9 · DOI · Paper page
- Michel van Kempen Stephanie S. Kim Charlotte Tumescheit Milot Mirdita Cameron L. M. Gilchrist Johannes Söding and Martin Steinegger. 2022. Foldseek: Fast and accurate protein structure search. Retrieved from 10.1101/2022.02.07.479398v1 · doi:10.1101/2022.02.07.479398v1
- Lei Wang, Hui Zhang, Wei Xu, Zhidong Xue, and Yan Wang. 2023. Deciphering the protein landscape with ProtFlash: A lightweight language model. Cell Rep. Phys. Sci. 4, 10 (2023), 101600. · DOI
- 10.1145/3307339.3342186 · DOI · Paper page
- Xin Wang, Xin Gao, Guohua Wang, and Dan Li. 2023. miProBERT: Identification of microRNA promoters based on the pre-trained model BERT. Brief. Bioinform. 24, 3 (2023), bbad093. · DOI
- Xi Wang, Ruichu Gu, Zhiyuan Chen, Yongge Li, Xiaohong Ji, Guolin Ke, and Han Wen. 2023. UNI-RNA: Universal pre-trained models revolutionize RNA research. Retrieved from https://www.biorxiv.org/content/2023.07.11.548588
- 10.3390/molecules28114430 · DOI
- 10.1038/s41598-022-10775-y · DOI · Paper page
- Zifeng Wang, Zichen Wang, Balasubramaniam Srinivasan, Vassilis N. Ioannidis, Huzefa Rangwala, and Rishita Anubhai. 2023. BioBridge: Bridging biomedical foundation models via knowledge graph. Retrieved from https://arXiv:2310.03320
- Zeyuan Wang, Qiang Zhang, Keyan Ding, Ming Qin, Xiang Zhuang, Xiaotong Li, and Huajun Chen. 2023. InstructProtein: Aligning human and protein language via knowledge instruction. Retrieved from https://arxiv.org/abs/2310.03269 · arXiv
- Zeyuan Wang, Qiang Zhang, Shuang-Wei HU, Haoran Yu, Xurui Jin, Zhichen Gong, and Huajun Chen. 2023. Multi-level protein structure pre-training via prompt learning. In Proceedings of the 11th International Conference on Learning Representations.
- Johannes Welbl, Nelson F. Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. Retrieved from https://arxiv.org/abs/1707.06209 · arXiv
- 10.1609/aaai.v37i4.25662 · DOI
- 10.1101/2024.05.14.594226v1 · doi:10.1101/2024.05.14.594226v1
- Jun Xia, Yanqiao Zhu, Yuanqi Du, and Stan Z. Li. 2022. A systematic survey of chemical pre-trained models. Retrieved from https://arxiv.org/abs/2210.16484 · arXiv
- Hongwang Xiao, Wenjun Lin, Hui Wang, Zheng Liu, and Qiwei Ye. [n.d.]. OPI: An open instruction dataset for adapting large language models to protein-related tasks. In Proceedings of the NeurIPS Workshop Foundation Models for Science: Progress, Opportunities, and Challenges.
- Teng Xiao, Chao Cui, Huaisheng Zhu, and Vasant G. Honavar. 2024. Molbind: Multimodal alignment of language, molecules, and proteins. Retrieved from https://arXiv:2403.08167
- Renchunzi Xie, Hongxin Wei, Lei Feng, and Bo An. 2022. Gearnet: Stepwise dual learning for weakly supervised domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8717–8725.
- Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang et al. 2023. Darwin series: Domain specific large language models for natural science. Retrieved from https://arxiv.org/abs/2308.13565 · arXiv
- Guoli Xiong, Zhenxing Wu, Jiacai Yi, Li Fu, Zhijiang Yang, Changyu Hsieh, Mingzhu Yin, Xiangxiang Zeng, Chengkun Wu, Aiping Lu et al. 2021. ADMETlab 2.0: An integrated online platform for accurate and comprehensive predictions of ADMET properties. Nucl. Acids Res. 49, W1 (2021), W5–W14.
- Hanwen Xu and Sheng Wang. 2022. ProTranslator: Zero-shot protein function prediction using textual description. In Proceedings of the International Conference on Research in Computational Molecular Biology. Springer, 279–294.
- Hanwen Xu, Addie Woicik, Hoifung Poon, Russ B. Altman, and Sheng Wang. 2023. Multilingual translation for zero-shot biomedical classification using BioTranslator. Nature Commun. 14, 1 (2023), 738. · DOI
- Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang. 2023. Protst: Multi-modality learning of protein sequences and biomedical texts. In Proceedings of the International Conference on Machine Learning. PMLR, 38749–38767.
- Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Ma Chang, Runcheng Liu, and Jian Tang. 2022. Peer: A comprehensive and multi-task benchmark for protein sequence understanding. Adv. Neural Inf. Process. Syst. 35 (2022), 35156–35173.
- Dongyu Xue, Han Zhang, Dongling Xiao, Yukang Gong, Guohui Chuai, Yu Sun, Hao Tian, Hua Wu, Yukun Li, and Qi Liu. 2020. X-MOL: Large-scale pre-training for molecular understanding and diverse molecular analysis. Retrieved from https://www.biorxiv.org/content/2020.12.23.424259
- 10.1038/s42256-022-00534-z · doi:10.1038/s42256-022-00534-z · Paper page
- Jianyi Yang, Ambrish Roy, and Yang Zhang. 2012. BioLiP: A semi-manually curated database for biologically relevant ligand–protein interactions. Nucl. Acids Res. 41, D1 (2012), D1096–D1103.
- Meng Yang Haiping Huang Lichao Huang Nan Zhang Jihong Wu Huanming Yang and Feng Mu. 2021. LOGO a contextualized pre-trained language model of human genome flexibly adapts to various downstream tasks by fine-tuning. https://www.researchgate.net/publication/354105080_LOGO_a_contextualized_pre-trained_language_model_of_human_genome_flexibly_adapts_to_various_downstream_tasks_by_fine-tuning · DOI
- Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2022. Linkbert: Pretraining language models with document links. Retrieved from https://arxiv.org/abs/2203.15827 · arXiv
- Chengxuan Ying, Mingqi Yang, Shuxin Zheng, Guolin Ke, Shengjie Luo, Tianle Cai, Chenglin Wu, Yuxin Wang, Yanming Shen, and Di He. 2021. First place solution of KDD Cup 2021 & OGB large-scale challenge graph prediction track. Retrieved from https://arxiv.org/abs/2106.08279 · arXiv
- Botao Yu, Frazier N. Baker, Ziqi Chen, Xia Ning, and Huan Sun. 2024. Llasmol: Advancing large language models for chemistry with a large-scale, comprehensive, high-quality instruction tuning dataset. Retrieved from https://arXiv:2402.09391
- Atakan Yüksel, Erva Ulusoy, Atabey Ünlü, and Tunca Doğan. 2023. Selformer: Molecular representation learning via selfies language models. Mach. Learn.: Sci. Technol. (2023).
- Barbara Zdrazil, Eloy Felix, Fiona Hunter, Emma J. Manners, James Blackshaw, Sybilla Corbett, Marleen de Veij, Harris Ioannidis, David Mendez Lopez, Juan F. Mosquera et al. 2023. The ChEMBL database in 2023: A drug discovery platform spanning multiple bioactivity data types and time periods. Nucl. Acids Res. (2023), gkad1004.
- Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia et al. 2022. Glm-130b: An open bilingual pre-trained model. Retrieved from https://arxiv.org/abs/2210.02414 · arXiv
- Haoyang Zeng, Matthew D. Edwards, Ge Liu, and David K. Gifford. 2016. Convolutional neural network architectures for predicting DNA–protein binding. Bioinformatics 32, 12 (2016), i121–i127.
- 10.1038/s41467-022-28494-3 · DOI
- Zheni Zeng, Bangchen Yin, Shipeng Wang, Jiarui Liu, Cheng Yang, Haishen Yao, Xingzhi Sun, Maosong Sun, Guotong Xie, and Zhiyuan Liu. 2023. Interactive molecular discovery with natural language. Retrieved from https://arxiv.org/abs/2306.11976 · arXiv
- Dan Zhang, Ziniu Hu, Sining Zhoubian, Zhengxiao Du, Kaiyu Yang, Zihan Wang, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. Sciglm: Training scientific language models with self-reflective instruction annotation and tuning. Retrieved from https://arXiv:2401.07950
- Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Dongzhan Zhou et al. 2024. Chemllm: A chemical large language model. Retrieved from https://arXiv:2402.06852
200 of 221 shown
Cited by
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
Wijk, H., Lin, T., Becker, J., Jawhar, S., Parikh, N., Broadley, T., Chan, L., Chen, M., Clymer, J., Dhyani, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.15114 · Cited by 187
Large language models transform biological research: from architecture to utilization
Wang, T., & Luo, Z. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4466-3 · Cited by 14
Considering the ethics of large machine learning models in the chemical sciences
Spotte-Smith, E. W. C. (2025) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/adec3c · Cited by 6
Zhang, S., Xie, L., Tiourine, D., & Xie, L. (2026) · Journal of Cheminformatics · doi:10.1186/s13321-026-01227-0 · Cited by 2
Metadata
- Authors
- Zhang, Q., Ding, K., Lv, T., Wang, X., Yin, Q., Zhang, Y., Yu, J., Wang, Y., Li, X., Xiang, Z., Zhuang, X., Wang, Z., Qin, M., Zhang, M., Zhang, J., Cui, J., Xu, R., Chen, H., Fan, X., Xing, H., & Chen, H.
- Venue
- ACM Computing Surveys
- Published
- 2025-02-10
- Volume
- 57
- Issue
- 6
- Pages
- 1-38
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4406828533
- Cited by
- 122
- References
- 95
- FWCI
- 47.85
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.