Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, Colin, Shazeer, Noam, Roberts, Adam, Lee, Katherine, Narang, Sharan, Matena, Michael, Zhou, Yanqi, Li, Wei +1 more·2019·arXiv
DOI 10.48550/arxiv.1910.10683

Cited by (11,035)

Items 11,001–11,035 of 11,035

  1. LENS: a transferable neuromuscular signature of musculoskeletal pain intensity
    Pedram Mouseli, Nathaniel W De Vera, Suha Sagheer · 2026 · bioRxiv (Cold Spring Harbor Laboratory)
  2. Enhancing Large Language Models for Medical Diagnosis in Pulmonology
    Rima Jemni, Rahma Boujelben, Lamia Hadrich Belguith · 2026 · Lecture notes in business information processing
  3. Evaluating Chinese large language models for HIV/AIDS health information: a multidimensional comparative study of response quality
    Jinhao Gu, Jianwei Liu, Yunrong Liu, Pingping Wang, DeHua Hu, Qizhen Zhu · 2026 · Frontiers in Public Health
  4. Sentence Simplification for Natural Language to Formal Logic Translation
    Roberto Milanese, Adam Pease, Richard Thompson · 2026 · Communications in computer and information science
  5. Tamarin (HSQ): Reversible Hierarchical KV-Cache Compression for LLM Inference
    Bazakov, Alikhan, Kiselev, Kirill · 2026
  6. AlignCheck: a Semantic Open-Domain Metric for Factual Consistency Assessment
    Aghaebrahimian, Ahmad · 2025 · arXiv
  7. Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts
    Veen, Dave Van, Uden, Cara Van, Blankemeier, Louis, Delbrouck, Jean-Benoit, Aali, Asad, Bluethgen, Christian, Pareek, Anuj, Polacin, Malgorzata, Reis, Eduardo Pontes, Seehofnerova, Anna et al. · 2023
  8. LittleBit: Ultra Low-Bit Quantization via Latent Factorization
    Lee, Banseok, Kim, Dongkyu, You, Youngcheon, Kim, Youngmin · 2025 · arXiv
  9. Learning Transferable Visual Models From Natural Language Supervision
    Radford, Alec, Kim, Jong Wook, Hallacy, Chris, Ramesh, Aditya, Goh, Gabriel, Agarwal, Sandhini, Sastry, Girish, Askell, Amanda, Mishkin, Pamela, Clark, Jack et al. · 2021 · arXiv
  10. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
    Lan, Zhenzhong, Chen, Mingda, Goodman, Sebastian, Gimpel, Kevin, Sharma, Piyush, Soricut, Radu · 2019 · arXiv
  11. HuggingFace's Transformers: State-of-the-art Natural Language Processing
    Wolf, Thomas, Debut, Lysandre, Sanh, Victor, Chaumond, Julien, Delangue, Clement, Moi, Anthony, Cistac, Pierric, Rault, Tim, Louf, Rémi, Funtowicz, Morgan et al. · 2019 · arXiv
  12. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
    Lewis, Patrick, Perez, Ethan, Piktus, Aleksandra, Petroni, Fabio, Karpukhin, Vladimir, Goyal, Naman, Küttler, Heinrich, Lewis, Mike, Yih, Wen-tau, Rocktäschel, Tim et al. · 2020 · arXiv
  13. FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
    Sohn, Kihyuk, Berthelot, David, Li, Chun-Liang, Zhang, Zizhao, Carlini, Nicholas, Cubuk, Ekin D., Kurakin, Alex, Zhang, Han, Raffel, Colin · 2020 · arXiv
  14. Big Bird: Transformers for Longer Sequences
    Zaheer, Manzil, Guruganesh, Guru, Dubey, Avinava, Ainslie, Joshua, Alberti, Chris, Ontanon, Santiago, Pham, Philip, Ravula, Anirudh, Wang, Qifan, Yang, Li et al. · 2020 · arXiv
  15. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
    Fedus, William, Zoph, Barret, Shazeer, Noam · 2021 · arXiv
  16. CoAtNet: Marrying Convolution and Attention for All Data Sizes
    Dai, Zihang, Liu, Hanxiao, Le, Quoc V., Tan, Mingxing · 2021 · arXiv
  17. Learning to summarize from human feedback
    Stiennon, Nisan, Ouyang, Long, Wu, Jeff, Ziegler, Daniel M., Lowe, Ryan, Voss, Chelsea, Radford, Alec, Amodei, Dario, Christiano, Paul · 2020 · arXiv
  18. REALM: Retrieval-Augmented Language Model Pre-Training
    Guu, Kelvin, Lee, Kenton, Tung, Zora, Pasupat, Panupong, Chang, Ming-Wei · 2020 · arXiv
  19. GraphCodeBERT: Pre-training Code Representations with Data Flow
    Guo, Daya, Ren, Shuo, Lu, Shuai, Feng, Zhangyin, Tang, Duyu, Liu, Shujie, Zhou, Long, Duan, Nan, Svyatkovskiy, Alexey, Fu, Shengyu et al. · 2020 · arXiv
  20. The Pile: An 800GB Dataset of Diverse Text for Language Modeling
    Gao, Leo, Biderman, Stella, Black, Sid, Golding, Laurence, Hoppe, Travis, Foster, Charles, Phang, Jason, He, Horace, Thite, Anish, Nabeshima, Noa et al. · 2021 · arXiv
  21. GLU Variants Improve Transformer
    Shazeer, Noam · 2020 · arXiv
  22. Long Range Arena: A Benchmark for Efficient Transformers
    Tay, Yi, Dehghani, Mostafa, Abnar, Samira, Shen, Yikang, Bahri, Dara, Pham, Philip, Rao, Jinfeng, Yang, Liu, Ruder, Sebastian, Metzler, Donald · 2020 · arXiv
  23. I-BERT: Integer-only BERT Quantization
    Kim, Sehoon, Gholami, Amir, Yao, Zhewei, Mahoney, Michael W., Keutzer, Kurt · 2021 · arXiv
  24. Self-supervised Pretraining of Visual Features in the Wild
    Goyal, Priya, Caron, Mathilde, Lefaudeux, Benjamin, Xu, Min, Wang, Pengchao, Pai, Vivek, Singh, Mannat, Liptchinsky, Vitaliy, Misra, Ishan, Joulin, Armand et al. · 2021 · arXiv
  25. Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
    Xiong, Wenhan, Li, Xiang Lorraine, Iyer, Srini, Du, Jingfei, Lewis, Patrick, Wang, William Yang, Mehdad, Yashar, Yih, Wen-tau, Riedel, Sebastian, Kiela, Douwe et al. · 2020 · arXiv
  26. BASE Layers: Simplifying Training of Large, Sparse Models
    Lewis, Mike, Bhosale, Shruti, Dettmers, Tim, Goyal, Naman, Zettlemoyer, Luke · 2021 · arXiv
  27. Generative Language Modeling for Automated Theorem Proving
    Polu, Stanislas, Sutskever, Ilya · 2020 · arXiv
  28. Recursively Summarizing Books with Human Feedback
    Wu, Jeff, Ouyang, Long, Ziegler, Daniel M., Stiennon, Nisan, Lowe, Ryan, Leike, Jan, Christiano, Paul · 2021 · arXiv
  29. VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning
    Tan, Hao, Lei, Jie, Wolf, Thomas, Bansal, Mohit · 2021 · arXiv
  30. BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
    Fries, Jason Alan, Weber, Leon, Seelam, Natasha, Altay, Gabriel, Datta, Debajyoti, Garda, Samuele, Kang, Myungsun, Su, Ruisi, Kusa, Wojciech, Cahyawijaya, Samuel et al. · 2022 · arXiv
  31. SpeechNet: A Universal Modularized Model for Speech Processing Tasks
    Chen, Yi-Chen, Chi, Po-Han, Yang, Shu-wen, Chang, Kai-Wei, Lin, Jheng-hao, Huang, Sung-Feng, Liu, Da-Rong, Liu, Chi-Liang, Lee, Cheng-Kuang, Lee, Hung-yi · 2021 · arXiv
  32. STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
    Dong, Peijie, Li, Lujun, Zhong, Yuedong, Du, Dayou, Fan, Ruibo, Chen, Yuhan, Tang, Zhenheng, Wang, Qiang, Xue, Wei, Guo, Yike et al. · 2024 · arXiv
  33. Solving Quantitative Reasoning Problems with Language Models
    Lewkowycz, Aitor, Andreassen, Anders, Dohan, David, Dyer, Ethan, Michalewski, Henryk, Ramasesh, Vinay, Slone, Ambrose, Anil, Cem, Schlag, Imanol, Gutman-Solo, Theo et al. · 2022 · arXiv
  34. Improving Code Autocompletion with Transfer Learning
    Zhou, Wen, Kim, Seohyun, Murali, Vijayaraghavan, Aye, Gareth Ari · 2021 · arXiv
  35. SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
    Singh, Amanpreet, D'Arcy, Mike, Cohan, Arman, Downey, Doug, Feldman, Sergey · 2022 · arXiv