Ruder, Sebastian

An overview of gradient descent optimization algorithms

Ruder, S. (2016) · arXiv

Stochastic Gradient Optimization TechniquesAdvanced Optimization Algorithms Research

Cited by 4,962

Universal Language Model Fine-tuning for Text Classification

Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 4,100

Gemini: A Family of Highly Capable Multimodal Models

Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv

Multimodal Machine Learning ApplicationsTopic Modeling

Cited by 990

PaLM 2 Technical Report

Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 834

To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks

Peters, M. E., Ruder, S., & Smith, N. A. (2019) · Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)

Domain Adaptation and Few-Shot LearningTopic Modeling

Cited by 432

Long Range Arena: A Benchmark for Efficient Transformers

Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., Rao, J., Yang, L., Ruder, S., & Metzler, D. (2020) · arXiv

Topic ModelingAdvanced Neural Network Applications

Cited by 241

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Rust, P., Pfeiffer, J., Vulić, I., Ruder, S., & Gurevych, I. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

Natural Language Processing TechniquesTopic Modeling

Cited by 202

Episodic Memory in Lifelong Language Learning

d'Autume, C. D. M., Ruder, S., Kong, L., & Yogatama, D. (2019) · arXiv

Domain Adaptation and Few-Shot LearningTopic Modeling

Cited by 95

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

Tay, Y., Tran, V. Q., Ruder, S., Gupta, J., Chung, H. W., Bahri, D., Qin, Z., Baumgartner, S., Yu, C., & Metzler, D. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 92

Mind the Gap: Assessing Temporal Generalization in Neural Language Models

Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Gimenez, M., d'Autume, C. D. M., Kocisky, T., Ruder, S., et al. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 90

Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution

Ruder, S., Ghaffari, P., & Breslin, J. G. (2016) · arXiv

Authorship Attribution and ProfilingHandwritten Text Recognition Techniques

Cited by 86

Language Models are Multilingual Chain-of-Thought Reasoners

Shi, F., Suzgun, M., Freitag, M., Wang, X., Srivats, S., Vosoughi, S., Chung, H. W., Tay, Y., Ruder, S., Zhou, D., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 60

LLM See, LLM Do: Leveraging Active Inheritance to Target Non-Differentiable Objectives

Shimabucoro, L., Ruder, S., Kreutzer, J., Fadaee, M., & Hooker, S. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

Topic ModelingEthics and Social Impacts of AI

Cited by 24

Latent Multi-Task Architecture Learning

Ruder, S., Bingel, J., Augenstein, I., & Søgaard, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence

Domain Adaptation and Few-Shot LearningTopic Modeling

Cited by 14

BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer

Asai, A., Kudugunta, S., Yu, X. V., Blevins, T., Gonen, H., Reid, M., Tsvetkov, Y., Ruder, S., & Hajishirzi, H. (2023) · arXiv

Domain Adaptation and Few-Shot LearningTopic Modeling

Cited by 10