Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research

Soldaini, Luca, Kinney, Rodney, Bhagia, Akshita · arXiv · 2024