Raffel, Colin

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 10,929

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., & Raffel, C. (2020) · arXiv

Machine Learning and AlgorithmsDomain Adaptation and Few-Shot Learning

Cited by 2,892

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer

Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Topic ModelingNatural Language Processing Techniques

Cited by 1,629

Emergent Abilities of Large Language Models

Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 1,264

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

Oliver, A., Odena, A., Raffel, C., Cubuk, E. D., & Goodfellow, I. J. (2018) · arXiv

Machine Learning and AlgorithmsDomain Adaptation and Few-Shot Learning

Cited by 772

Extracting Training Data from Large Language Models

Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv

Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in Data

Cited by 727

How Much Knowledge Can You Pack into the Parameters of a Language Model?

Roberts, A., Raffel, C., & Shazeer, N. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Topic ModelingInformation Retrieval and Search Behavior

Cited by 638

Multitask Prompted Training Enables Zero-Shot Task Generalization

Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 612

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 569

Crosslingual Generalization through Multitask Finetuning

Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M. S., Shen, S., Yong, Z. X., Schoelkopf, H., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 361

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models

Xue, L., Barua, A., Constant, N., Al-Rfou, R., Narang, S., Kale, M., Roberts, A., & Raffel, C. (2022) · Transactions of the Association for Computational Linguistics

Topic ModelingNatural Language Processing Techniques

Cited by 330

PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts

Bach, S., Sanh, V., Yong, Z. X., Webson, A., Raffel, C., Nayak, N. V., Sharma, A., Kim, T., Bari, M. S., Fevry, T., et al. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations

Natural Language Processing TechniquesTopic Modeling

Cited by 178

Large Language Models Struggle to Learn Long-Tail Knowledge

Kandpal, N., Deng, H., Roberts, A., Wallace, E., & Raffel, C. (2022) · arXiv

Topic ModelingInformation Retrieval and Search Behavior

Cited by 143

Ties-merging: Resolving interference when merging models

Yadav, P., Tam, D., Choshen, L., Raffel, C., & Bansal, M. (2023) · Advances in Neural Information Processing Systems 36

Model-Driven Software Engineering TechniquesFormal Methods in Verification

Cited by 95

Deduplicating Training Data Mitigates Privacy Risks in Language Models

Kandpal, N., Wallace, E., & Raffel, C. (2022) · arXiv

Privacy-Preserving Technologies in DataAdversarial Robustness in Machine Learning

Cited by 68

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Penedo, G., Kydlíček, H., allal, L. B., Lozhkov, A., Mitchell, M., Raffel, C., Von Werra, L., & Wolf, T. (2024) · arXiv

Topic ModelingComputational and Text Analysis Methods

Cited by 53

Petals: Collaborative Inference and Fine-tuning of Large Models

Borzunov, A., Baranchuk, D., Dettmers, T., Riabinin, M., Belkada, Y., Chumachenko, A., Samygin, P., & Raffel, C. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)

Topic ModelingNatural Language Processing Techniques

Cited by 52

Scaling Data-Constrained Language Models

Muennighoff, N., Rush, A. M., Barak, B., Scao, T. L., Piktus, A., Tazi, N., Pyysalo, S., Wolf, T., & Raffel, C. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 49

Scaling Data-Constrained Language Models

Muennighoff, N., Rush, A., Barak, B., Scao, T. L., Tazi, N., Piktus, A., Pyysalo, S., Wolf, T., & Raffel, C. (2023) · Advances in Neural Information Processing Systems 36

Topic ModelingNatural Language Processing Techniques

Cited by 47

Distributed Inference and Fine-tuning of Large Language Models Over The Internet

Borzunov, A., Ryabinin, M., Chumachenko, A., Baranchuk, D., Dettmers, T., Belkada, Y., Samygin, P., & Raffel, C. (2023) · arXiv

Topic ModelingPrivacy-Preserving Technologies in Data

Cited by 35

What Language Model to Train if You Have One Million GPU Hours?

Le Scao, T., Wang, T., Hesslow, D., Bekman, S., Bari, M. S., Biderman, S., Elsahar, H., Muennighoff, N., Phang, J., Press, O., et al. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022

Topic ModelingNatural Language Processing Techniques

Cited by 33

Crosslingual Generalization through Multitask Finetuning

Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Scao, T. L., Bari, M. S., Shen, S., Yong, Z.-X., Schoelkopf, H., et al. (2022) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 29

PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts

Bach, S. H., Sanh, V., Yong, Z.-X., Webson, A., Raffel, C., Nayak, N. V., Sharma, A., Kim, T., Bari, M. S., Fevry, T., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 21

SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model

Allal, L. B., Lozhkov, A., Bakouch, E., Blázquez, G. M., Penedo, G., Tunstall, L., Marafioti, A., Kydlíček, H., Lajarín, A. P., Srivastav, V., et al. (2025) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 13