Penedo, Guilherme

The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

Penedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Cappelli, A., Alobeidli, H., Pannier, B., Almazrouei, E., & Launay, J. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 164

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Penedo, G., Kydlíček, H., allal, L. B., Lozhkov, A., Mitchell, M., Raffel, C., Von Werra, L., & Wolf, T. (2024) · arXiv

Topic ModelingComputational and Text Analysis Methods

Cited by 53

The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only

Penedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Alobeidli, H., Cappelli, A., Pannier, B., Almazrouei, E., & Launay, J. (2023) · Advances in Neural Information Processing Systems 36

Topic ModelingNatural Language Processing Techniques

Cited by 25

SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model

Allal, L. B., Lozhkov, A., Bakouch, E., Blázquez, G. M., Penedo, G., Tunstall, L., Marafioti, A., Kydlíček, H., Lajarín, A. P., Srivastav, V., et al. (2025) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 13

FineWeb2: One Pipeline to Scale Them All - Adapting Pre-Training Data Processing to Every Language

Penedo, G., Kydlíček, H., Sabolčec, V., Messmer, B., Foroutan, N., Kargaran, A. H., Raffel, C., Jaggi, M., Von Werra, L., & Wolf, T. (2025) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 0