Tay, Yi

PaLM: Scaling Language Modeling with Pathways

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 2,590

Scaling Instruction-Finetuned Language Models

Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 1,441

Efficient transformers: A survey

Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys

Topic ModelingAdvanced Neural Network Applications

Cited by 1,267

Efficient Transformers: A Survey

Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2020) · arXiv

Topic ModelingAdvanced Neural Network Applications

Cited by 1,267

Emergent Abilities of Large Language Models

Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 1,264

PaLM 2 Technical Report

Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 834

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q., Chi, E., Zhou, D., et al. (2023) · Findings of the Association for Computational Linguistics: ACL 2023

Topic ModelingNatural Language Processing Techniques

Cited by 253

Synthesizer: Rethinking Self-Attention in Transformer Models

Tay, Y., Bahri, D., Metzler, D., Juan, D.-C., Zhao, Z., & Zheng, C. (2020) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 251

Long Range Arena: A Benchmark for Efficient Transformers

Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., Rao, J., Yang, L., Ruder, S., & Metzler, D. (2020) · arXiv

Topic ModelingAdvanced Neural Network Applications

Cited by 241

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

Lees, A., Tran, V. Q., Tay, Y., Sorensen, J., Gupta, J., Metzler, D., & Vasserman, L. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Hate Speech and Cyberbullying DetectionSentiment Analysis and Opinion Mining

Cited by 155

Transformer Memory as a Differentiable Search Index

Tay, Y., Tran, V., Dehghani, M., Ni, J., Bahri, D., Mehta, H., Qin, Z., Hui, K., Zhao, Z., Gupta, J., et al. (2022) · Advances in Neural Information Processing Systems 35

Topic ModelingInformation Retrieval and Search Behavior

Cited by 128

Rethinking search: making domain experts out of dilettantes

Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · ACM SIGIR Forum

Information Retrieval and Search BehaviorTopic Modeling

Cited by 112

Scaling vision transformers to 22 billion parameters

Dehghani, M., Djolonga, J., Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A., Caron, M., Geirhos, R., Alabdulmohsin, I., et al. (2023) · arXiv

Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

Cited by 112

Larger language models do in-context learning differently

Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., et al. (2023) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 109

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

Tay, Y., Tran, V. Q., Ruder, S., Gupta, J., Chung, H. W., Bahri, D., Qin, Z., Baumgartner, S., Yu, C., & Metzler, D. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 92

Confident adaptive language modeling

Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Bahri, D., Tran, V., Tay, Y., & Metzler, D. (2022) · Advances in Neural Information Processing Systems 35

Natural Language Processing TechniquesSpeech Recognition and Synthesis

Cited by 91

Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers

Tay, Y., Dehghani, M., Rao, J., Fedus, W., Abnar, S., Chung, H. W., Narang, S., Yogatama, D., Vaswani, A., & Metzler, D. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 63

Language Models are Multilingual Chain-of-Thought Reasoners

Shi, F., Suzgun, M., Freitag, M., Wang, X., Srivats, S., Vosoughi, S., Chung, H. W., Tay, Y., Ruder, S., Zhou, D., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 60

The Benchmark Lottery

Dehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv

Machine Learning and Data ClassificationMachine Learning and Algorithms

Cited by 48

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 44

Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?

Tay, Y., Dehghani, M., Abnar, S., Chung, H., Fedus, W., Rao, J., Narang, S., Tran, V., Yogatama, D., & Metzler, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023

Topic ModelingNatural Language Processing Techniques

Cited by 34

Recitation-Augmented Language Models

Sun, Z., Wang, X., Tay, Y., Yang, Y., & Zhou, D. (2022) · arXiv

Topic ModelingMultimodal Machine Learning Applications

Cited by 34

StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling

Shen, Y., Tay, Y., Zheng, C., Bahri, D., Metzler, D., & Courville, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

Natural Language Processing TechniquesTopic Modeling

Cited by 29

Inverse Scaling Can Become U-Shaped

Wei, J., Kim, N., Tay, Y., & Le, Q. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

Topic ModelingNatural Language Processing Techniques

Cited by 25