Biderman, Stella

Multitask Prompted Training Enables Zero-Shot Task Generalization

Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 612

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 569

The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 492

OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Ahdritz, G., Bouatta, N., Floristean, C., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T. J., Berenberg, D., Fisk, I., Zanichelli, N., et al. (2024) · Nature Methods

Protein Structure and DynamicsMachine Learning in Bioinformatics

Cited by 439

RWKV: Reinventing RNNs for the Transformer Era

Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Derczynski, L., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023

Topic ModelingNatural Language Processing Techniques

Cited by 390

Crosslingual Generalization through Multitask Finetuning

Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M. S., Shen, S., Yong, Z. X., Schoelkopf, H., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 361

GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow

Black, S., Leo, G., Wang, P., Leahy, C., & Biderman, S. (2021) · Zenodo

Topic ModelingNatural Language Processing Techniques

Cited by 324

Open Problems in Mechanistic Interpretability

Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv

Explainable Artificial Intelligence (XAI)Neural Networks and Applications

Cited by 291

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2021) · arXiv

Natural Language Processing TechniquesTopic Modeling

Cited by 179

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2022) · Transactions of the Association for Computational Linguistics

Natural Language Processing TechniquesTopic Modeling

Cited by 179

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Biderman, S., Schoelkopf, H., Anthony, Q., Bradley, H., O'Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 160

On the Societal Impact of Open Foundation Models

Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., Hopkins, A., Bankston, K., Biderman, S., Bogen, M., et al. (2024) · arXiv

Ethics and Social Impacts of AILaw, AI, and Intellectual Property

Cited by 103

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

Laurençon, H., Saulnier, L., Wang, T., Akiki, C., del Moral, A. V., Scao, T. L., Von Werra, L., Mou, C., Ponferrada, E. G., Nguyen, H., et al. (2023) · arXiv

Natural Language Processing TechniquesTopic Modeling

Cited by 93

Data Governance in the Age of Large-Scale Data-Driven Language Technology

Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency

Ethics and Social Impacts of AIDigitalization, Law, and Regulation

Cited by 75

GPT-NeoX-20B: An Open-Source Autoregressive Language Model

Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 67

The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs

Ruis, L., Khan, A., Biderman, S., Hooker, S., Rocktäschel, T., & Grefenstette, E. (2023) · Advances in Neural Information Processing Systems 36

Natural Language Processing TechniquesSyntax, Semantics, Linguistic Variation

Cited by 47

LEACE: Perfect linear concept erasure in closed form

Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., & Biderman, S. (2023) · arXiv

Cryptography and Data SecurityPrivacy-Preserving Technologies in Data

Cited by 44

Fooling MOSS Detection with Pretrained Language Models

Biderman, S., & Raff, E. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Academic integrity and plagiarismSoftware Engineering Research

Cited by 37

Fooling MOSS Detection with Pretrained Language Models

Biderman, S., & Raff, E. (2022) · arXiv

Academic integrity and plagiarismSoftware Engineering Research

Cited by 37

What Language Model to Train if You Have One Million GPU Hours?

Le Scao, T., Wang, T., Hesslow, D., Bekman, S., Bari, M. S., Biderman, S., Elsahar, H., Muennighoff, N., Phang, J., Press, O., et al. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022

Topic ModelingNatural Language Processing Techniques

Cited by 33

BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting

Yong, Z. X., Schoelkopf, H., Muennighoff, N., Aji, A. F., Adelani, D. I., Almubarak, K., Bari, M. S., Sutawika, L., Kasai, J., Baruwa, A., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingNatural Language Processing Techniques

Cited by 30

Crosslingual Generalization through Multitask Finetuning

Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Scao, T. L., Bari, M. S., Shen, S., Yong, Z.-X., Schoelkopf, H., et al. (2022) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 29

Eliciting Latent Predictions from Transformers with the Tuned Lens

Belrose, N., Ostrovsky, I., McKinney, L., Furman, Z., Smith, L., Halawi, D., Biderman, S., & Steinhardt, J. (2023) · arXiv

Explainable Artificial Intelligence (XAI)Adversarial Robustness in Machine Learning

Cited by 27

RWKV: Reinventing RNNs for the Transformer Era

Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Grella, M., et al. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 17