The DOI matches its Crossref record; 1 of the 17 references it cites were found in a registry or an index, and 16 are not checked yet.
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Open accessAbstract
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Evan Walsh, Luke Zettlemoyer, Noah Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Of these models, only OLMo is accompanied by public disclosure of its training data (Soldaini et al., 2024;Groeneveld et al., 2024).”
From HALoGEN: Fantastic LLM Hallucinations and Where to Find Them (2025) · cites this paper as Soldaini et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Across the literature, a wide range of filtering strategies have consistently improved overall dataset quality”
From OpenThoughts: Data Recipes for Reasoning Models (2025) · cites this paper as Soldaini et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 41
- S. Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David M. Mimno, and Daphne Ippolito. 2023. A pretrainer's guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. ArXiv, abs/2305.13169. · arXiv
- Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hannaneh Ha- jishirzi, Noah A Smith, Kyle Richardson, and Jesse Dodge. 2023. Paloma: A benchmark for evaluating language model fit. arXiv [cs.CL].
- Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Tree- bank: Annotating predicate argument structure. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
- Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zam- parelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 216-223, Reykjavik, Iceland. European Lan- guage Resources Association (ELRA).
- Marc Marone and Benjamin Van Durme. 2023. Data portraits: Recording foundation model training data. ArXiv, abs/2303.03919. · arXiv
- Srdjan Matic, Costas Iordanou, Georgios Smaragdakis, and Nikolaos Laoutaris. 2020. Identifying sensitive urls at web-scale. Proceedings of the ACM Internet Measurement Conference.
- Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer Sentinel Mixture Mod- els. arXiv preprint arXiv:1609.07843. · arXiv
- Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct elec- tricity? a new dataset for open book question answer- ing. arXiv [cs.CL].
- Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Long- pre. 2023a. Octopack: Instruction tuning code large language models. arXiv preprint arXiv:2308.07124. · arXiv
- Ofir Press, Noah A Smith, and Mike Lewis. 2021. Train short, test long: Attention with linear biases enables input length extrapolation. Open Data Commons. 2010. Open Data Commons Attribution License (ODC-By) v1.0. https:// opendatacommons.org/licenses/by/1-0/. An- nouncement. [accessed August 2023].
- OpenAI. 2023. Gpt-4 technical report. ArXiv, abs/2303.08774. · arXiv
- Antonis Papasavva, Savvas Zannettou, Emiliano De Cristofaro, Gianluca Stringhini, and Jeremy Blackburn. 2020. Raiders of the lost kek: 3.5 years of augmented 4chan posts from the politically incorrect board. 14th International AAAI Conference On Web And Social Media (ICWSM), 2020.
- Guilherme Penedo, Quentin Malartic, Daniel Hess- low, Ruxandra-Aimée Cojocaru, Alessandro Cap- pelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The refined- web dataset for falcon llm: Outperforming curated corpora with web data, and web data only. ArXiv, abs/2306.01116. · arXiv
- Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurençon, Gérard Dupont, Sasha Luccioni, Yacine Jernite, and Anna Rogers. 2023. The ROOTS search tool: Data transparency for LLMs. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 304-314, Toronto, Canada. Association for Computational Linguistics.
- Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. Wic: the word-in-context dataset for evaluat- ing context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1267-1273.
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susan- nah Young, Eliza Rutherford, Tom Hennigan, Ja- cob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Mari- beth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John F. J. Mellor, Irina Higgins, Antonia Creswell, Nathan McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, L. Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsim- poukelli, N. K. Grigorev, Doug Fritz, Thibault Sotti- aux, Mantas Pajarskas, Tobias Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew G. Johnson, Blake A. Hechtman, Laura Weidinger, Iason Gabriel, William S. Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem W. Ayoub, Jeff Stanway, L. L. Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. 2021. Scal- ing language models: Methods, analysis & insights from training gopher. ArXiv, abs/2112.11446. · arXiv
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Trans- former. The Journal of Machine Learning Research, 21(1):5485-5551.
- Anand Rajaraman and Jeffrey David Ullman. 2011. Mining of Massive Datasets. Cambridge University Press, USA.
- Yasaman Razeghi, Robert L Logan IV, Matt Gardner, and Sameer Singh. 2022. Impact of pretraining term frequencies on few-shot numerical reasoning. In Findings of the Association for Computational Lin- guistics: EMNLP 2022, pages 840-854, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Machel Reid, Victor Zhong, Suchin Gururangan, and Luke Zettlemoyer. 2022. M2D2: A massively multi- domain language modeling dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 964-975, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
- Manoel Horta Ribeiro, Jeremy Blackburn, Barry Brad- lyn, Emiliano De Cristofaro, Gianluca Stringhini, Summer Long, Stephanie Greenberg, and Savvas Zan- nettou. 2021. The evolution of the manosphere across the web. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, pages 196-207.
- Melissa Roemmele, Cosmin Adrian Bejan, and An- drew S Gordon. 2011. Choice of plausible alterna- tives: An evaluation of commonsense causal reason- ing. In 2011 AAAI Spring Symposium Series.
- Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhaga- vatula, and Yejin Choi. 2019. WinoGrande: An ad- versarial winograd schema challenge at scale. arXiv [cs.CL].
- Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A Smith. 2021. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. arXiv [cs.CL].
- Preethi Seshadri, Sameer Singh, and Yanai Elazar. 2023. The bias amplification paradox in text-to-image gen- eration. arXiv preprint arXiv:2308.00755. · arXiv
- Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Ro- man Novak, Roman Sitelew, Ronan Le Bras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Russ Salakhutdinov, Ryan Andrew Chi, Seungjae Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mo- hammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bow- man, Samuel Stern Schoenholz, Sanghyun Han, San- jeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixi- ang Shane Gu, Shubh Pachchigar, Shubham Tosh- niwal, Shyam Upadhyay, Shyamolima Shammie Debnath, Siamak Shakeri, Simon Thormeyer, Si- mone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven Piantadosi, Stuart Shieber, Summer Mish- erghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsunori Hashimoto, Te-Lin Wu, Théo Desbor- des, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Ko- rnev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victo- ria Nyamai, Vikas Raunak, Vinay Venkatesh Ra- masesh, vinay uday prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadol- lah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, and Ziyi Wu. 2023. Be- yond the imitation game: Quantifying and extrapolat- ing the capabilities of language models. Transactions on Machine Learning Research.
- Nishant Subramani, Sasha Luccioni, Jesse Dodge, and Margaret Mitchell. 2023. Detecting personal informa- tion in training corpora: an analysis. In Proceedings of the 3rd Workshop on Trustworthy Natural Lan- guage Processing (TrustNLP 2023), pages 208-220, Toronto, Canada. Association for Computational Lin- guistics.
- Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, Yaguang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, De- hao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kath- leen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Ale- jandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Co- hen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022. LaMDA: Language models for dialog ap- plications. arXiv [cs.CL].
- Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari S. Morcos. 2023. D4: Improving llm pretrain- ing via document de-duplication and diversification. ArXiv, abs/2308.12284. · arXiv
- Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. Llama: Open and efficient foundation language models. ArXiv, abs/2302.13971. · arXiv
- Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hos- seini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, An- gela Fan, Melanie Kambadur, Sharan Narang, Aure- lien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models. ArXiv, abs/2307.09288. · arXiv
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Aman- preet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understand- ing systems. Advances in neural information process- ing systems, 32.
- Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. arXiv [cs.HC].
- Guillaume Wenzek, Marie-Anne Lachaux, Alexis Con- neau, Vishrav Chaudhary, Francisco Guzmán, Ar- mand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 4003-4012, Marseille, France. European Language Resources Association.
- Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. 2015. Towards ai-complete ques- tion answering: A set of prerequisite toy tasks. arXiv: Artificial Intelligence.
- Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mT5: A massively multilingual pre-trained text-to-text transformer. arXiv [cs.CL].
- Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, Joseph E. Gonzalez, and Ion Stoica. 2023. Rethinking bench- mark and contamination for language models with rephrased samples. ArXiv, abs/2311.04850. · arXiv
- Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
- Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Es- ther Cheng, Jie Liu, Qunshu Lin, Raven Yuan, Tuney Zheng, Wei Pang, Xinrun Du, Yiming Liang, Yinghao Ma, Yizhi Li, Ziyang Ma, Bill Lin, Emmanouil Bene- tos, Huan Yang, Junting Zhou, Kaijing Ma, Minghao Liu, Morry Niu, Noah Wang, Quehry Que, Ruibo Liu, Sine Liu, Shawn Guo, Soren Gao, Wangchunshu Zhou, Xinyue Zhang, Yizhi Zhou, Yubo Wang, Yuelin Bai, Yuhan Zhang, Yuxiang Zhang, Zenith Wang, Zhenzhu Yang, Zijian Zhao, Jiajun Zhang, Wanli Ouyang, Wenhao Huang, and Wenhu Chen. 2024. Map-neo: Highly capable and transparent bilingual large language model series. arXiv preprint arXiv: 2405.19327. · arXiv
- Hao Zhang. 2022. Language model decomposition: Quantifying the dependency and correlation of lan- guage models. In Proceedings of the 2022 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 2508-2517, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Terry Yue Zhuo, Armel Zebaze, Nitchakarn Suppat- tarachai, Leandro von Werra, Harm de Vries, Qian Liu, and Niklas Muennighoff. 2024. Astraios: Parameter-efficient instruction tuning code large lan- guage models. arXiv preprint arXiv:2401.00788. · arXiv
Cited by
Sorted by citations
Datasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 268
Continual Learning of Large Language Models: A Comprehensive Survey
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2025) · ACM Computing Surveys · doi:10.1145/3735633 · Cited by 291
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Longpre, S., Yauney, G., Reif, E., Lee, K., Roberts, A., Zoph, B., Zhou, D., Wei, J., Robinson, K., Mimno, D., et al. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.179 · Cited by 168
A survey on large language models with multilingualism: recent advances and new frontiers
Huang, K., Mo, F., Zhang, X., Li, H., Li, Y., Zhang, Y., yi, W., Mao, Y., Liu, J., Xu, Y., et al. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11534-5 · Cited by 131
Lamaakal, I., Maleh, Y., El Makkaoui, K., Ouahbi, I., Pławiak, P., Alfarraj, O., Almousa, M., & Abd El-Latif, A. A. (2025) · Sensors · doi:10.3390/s25051318 · Cited by 60
Large language models present new questions for decision support
Handler, A., Larsen, K. R. T., & Hackathorn, R. D. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2024.102811 · Cited by 75
Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
Wang, R., Ribeiro, A., Robinson, C., Loeb, S., & Demszky, D. (2024) · doi:10.21203/rs.3.rs-5363154/v1 · Cited by 53
Wang, R., Gao, Z., Zhang, L., Yue, S., & Gao, Z. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100755 · Cited by 61
Synthesizing scientific literature with retrieval-augmented language models
Asai, A., He, J., Shao, R., Shi, W., Singh, A., Chang, J. C., Lo, K., Soldaini, L., Feldman, S., D’Arcy, M., et al. (2026) · Nature · doi:10.1038/s41586-025-10072-4 · Cited by 38
Self-citationLarge language models for batteries
Zuo, W., Zheng, H., He, T., Vishwanath, V., Chan, M. K. Y., Stevens, R. L., Amine, K., & Xu, G. (2025) · Joule · doi:10.1016/j.joule.2025.102037 · Cited by 35
DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMs
Mousavi, S. M., Alghisi, S., & Riccardi, G. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.471 · Cited by 64
Tucano: Advancing neural text generation for Portuguese
Corrêa, N. K., Sen, A., Falk, S., & Fatimah, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101325 · Cited by 36
User Privacy and Large Language Models: An Analysis of Frontier Developers’ Privacy Policies
King, J., Klyman, K., Capstick, E., Saade, T., & Hsieh, V. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i2.36646 · Cited by 21
An archival perspective on pretraining data
Desai, M. A., Pasquetto, I. V., Jacobs, A. Z., & Card, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100966 · Cited by 26
What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners' Perspective
Yu, X., Zhang, Z., Niu, F., Hu, X., Xia, X., & Grundy, J. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695061 · Cited by 27
Can LLM Graph Reasoning Generalize beyond Pattern Memorization?
Zhang, Y., Wang, H., Feng, S., Tan, Z., Han, X., He, T., & Tsvetkov, Y. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.127 · Cited by 21
The 2025 Foundation Model Transparency Index
Wan, A., Klyman, K., Kapoor, S., Maslej, N., Longpre, S., Xiong, B., Liang, P., & Bommasani, R. (2026) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i4.17165 · Cited by 39
OctoPack: Instruction Tuning Code Large Language Models
Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T. Y., Singh, S., Tang, X., von Werra, L., & Longpre, S. (2023) · arXiv · doi:10.48550/arxiv.2308.07124 · Cited by 245
Self-citationLatxa: An Open Language Model and Evaluation Suite for Basque
Etxaniz, J., Sainz, O., Miguel, N. P., Aldabe, I., Rigau, G., Agirre, E., Ormazabal, A., Artetxe, M., & Soroa, A. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.799 · Cited by 30
Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets
Walsh, M., Antoniak, M., & Preus, A. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.914 · Cited by 17
Blind Baselines Beat Membership Inference Attacks for Foundation Models
Das, D., Zhang, J., & Tramèr, F. (2025) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw67851.2025.00016 · Cited by 17
Creative Writers' Attitudes on Writing as Training Data for Large Language Models
Gero, K. I., Desai, M., Schnitzler, C., Eom, N., Cushman, J., & Glassman, E. L. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713287 · Cited by 16
Intellectual property issues in artificial intelligence trained on scraped data
OECD (2025) · OECD artificial intelligence papers · doi:10.1787/d5241a23-en · Cited by 19
Leça, M. D. M., Valença, L., Santos, R., & Santos, R. D. S. (2025) · IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE) · doi:10.1109/wsese66602.2025.00013 · Cited by 14
Benchmarking Linguistic Diversity of Large Language Models
Guo, Y., Shang, G., & Clavel, C. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.47 · Cited by 11
A Review of the Challenges with Massive Web-Mined Corpora Used in Large Language Models Pre-training
Perełkiewicz, M., & Poświata, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81596-6_14 · Cited by 16
HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights
Gökdemir, O., Siebenschuh, C., Brace, A., Wells, A. I., Hsu, B., Hippe, K., Setty, P. V., Ajith, A., Pauloski, J. G., Sastry, V. K., et al. (2025) · Platform for Advanced Scientific Computing Conference (PASC) · doi:10.1145/3732775.3733586 · Cited by 17
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.71 · Cited by 13
Self-citation5G INSTRUCT Forge: An Advanced Data Engineering Pipeline for Making LLMs Learn 5G
Said, A. I. A., Mekrache, A., Boutiba, K., Ramantas, K., Ksentini, A., & Rahmani, M. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3516055 · Cited by 10
Croskey, P., Offert, F., Jacobs, J., & Thaler, K. M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732058 · Cited by 10
Detection and Measurement of Syntactic Templates in Generated Text
Shaib, C., Elazar, Y., Li, J. J., & Wallace, B. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.368 · Cited by 8
Self-citationZhang, Y., Li, S., Qian, C., Liu, J., Yu, P., Han, C., Fung, Y. R., McKeown, K. R., Zhai, C., Li, M., et al. (2025) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2025.fever-1.10 · Cited by 7
Generative Aesthetics: On formal stuckness in AI verse
Heuser, R. (2025) · Journal of Cultural Analytics · doi:10.22148/001c.144825 · Cited by 7
Clavié, B., Cooper, N., & Warner, B. C. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100150 · Cited by 10
Task Oriented In-Domain Data Augmentation
Liang, X. G., Hu, X., Zuo, S., Gong, Y., Lou, Q., Liu, Y., Huang, S., & Jiao, J. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1162 · Cited by 12
Towards More Realistic Extraction Attacks: An Adversarial Perspective
More, Y., Ganesh, P., & Farnadi, G. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.62 · Cited by 6
Detecting Training Data of Large Language Models via Expectation Maximization
Kim, G., Li, Y., Spiliopoulou, E., Ma, J., & Wang, W. Y. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.49 · Cited by 14
Dolomites : Domain-Specific Long-Form Methodical Tasks
Malaviya, C., Agrawal, P., Ganchev, K., Srinivasan, P., Huot, F., Berant, J., Yatskar, M., Das, D., Lapata, M., & Alberti, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00727 · Cited by 8
Shades of zero: Distinguishing impossibility from inconceivability
Hu, J., Sosa, F., & Ullman, T. D. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104640 · Cited by 8
LongAttn: Selecting Long-context Training Data via Token-level Attention
Wu, L., Zhu, D., Zhao, G., Yu, Z., Ran, J., Wong, X., Sun, L., & Li, S. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.991 · Cited by 12
Sound Check: Auditing Recent Audio Dataset Practices
Agnew, W., Barnett, J., Chu, A.-N., Hong, R., Feffer, M., Netzorg, R., Jiang, H. H., Awumey, E., & Das, S. (2025) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v8i1.36528 · Cited by 3
Dissecting Fine-Tuning Unlearning in Large Language Models
Hong, Y., Zou, Y., Hu, L., Zeng, Z., Wang, D., & Yang, H. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.228 · Cited by 5
Metadata Conditioning Accelerates Language Model Pre-trainingPreprint
Gao, T., Wettig, A., He, L., Dong, Y., Malladi, S., & Chen, D. (2025) · Qeios · doi:10.32388/d226qh · Cited by 22
Bolaños-Burgos, F., Salatino, A. A., Osborne, F., & Motta, E. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3921 · Cited by 6
Towards Collecting Royalties for Copyrighted Data for Generative Models
Ludwig, H., Zhou, Y., Zawad, S., Ong, Y. J., Li, P., Butler, E., & Zahid, E. (2024) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws62655.2024.00020 · Cited by 5
Data-Prep-Kit: getting your data ready for LLM application development
Wood, D., Lublinsky, B., Roytman, A., Singh, S., Adam, C., Adebayo, A. A., An, S., Chang, Y. C., Dang, X. H., Desai, N. V., et al. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825085 · Cited by 4
Self-Rationalization in the Wild: A Large-scale Out-of-Distribution Evaluation on NLI-related tasks
Yang, J., Glockner, M., Rocha, A., & Gurevych, I. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00741 · Cited by 4
Nielsen, J., Schneider–Kamp, P., & Galke, L. P. A. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.694 · Cited by 5
The KL3M Data Project: Copyright-Clean Training Resources for Large Language ModelsPreprint
Bommarito, M. J., Bommarito, J., & Katz, D. M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5211933 · Cited by 10
Trustworthiness of large language models: hallucinations
Brunello, N. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-33592-1.00007-3 · Cited by 4
Tseng, Y., Parcollet, T., van Dalen, R. C., Zhang, S., & Bhattacharya, S. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434665 · Cited by 3
Sun, K., & Dredze, M. H. (2025) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2025.repl4nlp-1.11 · Cited by 6
ComPO: Community Preferences for Language Model Personalization
Kumar, S., Park, C. Y., Tsvetkov, Y., Smith, N. A., & Hajishirzi, H. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.419 · Cited by 3
Self-citationMeta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
Zhuang, X., Peng, J., Ma, R., Wang, Y., Bai, T., Wei, X., Jiantao, Q., Zhang, C., Qian, Y., & He, C. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.533 · Cited by 3
Malicious and Unintentional Disclosure Risks in Large Language Models for Code Generation
Rabin, M. R. I., McGregor, S., & Judd, N. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C) · doi:10.1109/saner-c66551.2025.00016 · Cited by 2
Detecting Training Data For Large Language Models: A Survey
Yang, C., Li, J., Shulin, L. A. N., Wang, Y., Du, H., Gong, C., Yao, X., Niyato, D. T., & Zhu, L. (2026) · ACM Computing Surveys · doi:10.1145/3779430 · Cited by 2
Data Preparation for Large Language Models
Liang, H., Wong, Z. H., Liu, R., Wang, Y.-H., Qiang, M.-Y., Zhao, Z.-Y., Shen, C.-Y., He, C.-H., Zhang, W.-T., & Cui, B. (2026) · Journal of Computer Science and Technology · doi:10.1007/s11390-026-5948-8 · Cited by 6
Position Paper: Data-Centric AI in the Age of Large Language Models
Xu, X., Wu, Z., Qiao, R., Verma, A., Shu, Y., Wang, J., Niu, X., He, Z., Chen, J., Zhou, Z., et al. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.695 · Cited by 3
What Does It Mean to “Misuse” Research Data?
Pasquetto, I. V., Thomer, A., Acker, A., Chtena, N., & Desai, M. A. (2024) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1108 · Cited by 3
Simplifying Scholarly Abstracts for Accessible Digital Libraries Using Language Models
Wang, H., & Clark, J. A. (2024) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3677389.3702490 · Cited by 2
Generative AI Tools for Systematic Approaches to Literature Review: Insights from Danish Librarians
Khalid, M. S., Moussa, A. M. E.-H., Wu, J., & Ekstrøm, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93567-1_20 · Cited by 2
Generative AI Models as Wicked Resources: A Dynamic Perspective on Resource Governance
Albayraktaroğlu, A., Mergen, A., & Güven, Ç. (2026) · Journal of Management · doi:10.1177/01492063261434193 · Cited by 2
Data, Data Everywhere: A Guide for Pretraining Dataset Construction
Parmar, J., Prabhumoye, S., Jennings, J., Liu, B., Jhunjhunwala, A., Wang, Z., Patwary, M. A., Shoeybi, M., & Catanzaro, B. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.596 · Cited by 9
A Unified Evaluation and Governance Framework for Trustworthy LLM AgentsPreprint
Bamil, V., & LINGAMGUNTA, R. K. K. (2026) · doi:10.36227/techrxiv.176799772.28164151/v1 · Cited by 3
OpenThoughts: Data Recipes for Reasoning Models
Guha, E., Marten, R., Keh, S., Raoof, N., Smyrnis, G., Bansal, H., Nezhurina, M., Mercat, J., Vu, T., Sprague, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.04178 · Cited by 1
Self-citationDeveloping and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models
Jiang, J., Truong, A. K. Y., Chen, Y., Bao, Q., Wang, S., Chen, P., Wang, J., Kong, L., Li, Y., & Wu, C. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.102 · Cited by 1
Data preparation for fine tuning Large Language Models
Selvam, P., Patel, H., Surendran, S., & Singh, S. (2024) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3703323.3704802 · Cited by 1
Perturb Your Data: Paraphrase-Guided Training Data Watermarking
Shetty, P., Haque, M., Babkin, P., Ma, Z., Liu, X., & Veloso, M. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i39.40575 · Cited by 1
Training-Free Open-Set Domain Adaptation With Vision-Language Models
Yu, Z., Lü, K., Wu, K., Chen, H., Li, F., & Li, J. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3684807 · Cited by 1
Expanding External Access to Frontier AI Models for Dangerous Capability Evaluations
Charnock, J., Tlaie, A., O'Brien, K., Casper, S., & Homewood, A. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812365 · Cited by 6
Benchmarking Document Parsers on Mathematical Formula Extraction from PDFs
Horn, P., & Keuper, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31583-0_7 · Cited by 1
Train and deploy: Expropriation and accumulation in the generative AI economy
Ravishankar, V., & Kienscherf, M. (2026) · Capital & Class · doi:10.1177/03098168261478525 · Cited by 1
Open Shouldn't Mean Exempt: Open Source Exceptionalism in Generative AIPreprint
Atkinson, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5355736 · Cited by 1
Self-citationDeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
Patel, L., Arabzadeh, N., Gupta, H., Sundar, A., Stoica, I., Zaharia, M., & Guestrin, C. (2025) · arXiv · doi:10.48550/arxiv.2508.20033
Nollywood: Let’s Go to the Movies!Preprint
Ortega, J. E., Chen, W., & Ahmad, I. S. (2024) · Preprints.org · doi:10.20944/preprints202402.0845.v1
Intellecta Cognitiva: A Comprehensive Dataset for Advancing Academic Knowledge and Machine ReasoningPreprint
Ditto, P. S., Ajmal, P. S., & Jithin, V. G. (2024) · Preprints.org · doi:10.20944/preprints202404.0849.v1
Guerrero, L. E. M., Ceballos, Y. F., & Rojas, L. D. T. (2024) · Journal of Southwest Jiaotong University · doi:10.35741/issn.0258-2724.59.2.26
Bella Turca: A Large-Scale Dataset of Diverse Text Sources for Turkish Language Modeling
Altinok, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70563-2_16
TAXI: Evaluating Categorical Knowledge Editing for Language Models
Powell, D., Gerych, W., & Hartvigsen, T. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.906
Understanding Creators' Acceptance of Content Reuse
Sinha, R., & Li, H. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681911
The SlayQA benchmark of social reasoning: testing gender-inclusive generalization with neopronouns
Bunzeck, B., & Zarrieß, S. (2024) · GenBench Workshop on Generalisation (Benchmarking) in NLP (GenBench) · doi:10.18653/v1/2024.genbench-1.3 · Cited by 1
3CBench: A Unified Benchmarking Framework for the Computing Capacity of Heterogeneous AI ClustersPreprint
Zhang, W., Wang, X., Yan, J., Feng, J., Liu, Y., Li, H., Chen, Q., Tang, Z. N., Cui, X. Y., & Yang, F. (2025) · Research Square · doi:10.21203/rs.3.rs-7602328/v1
ULTRABENCH: Benchmarking LLMs under Extreme Fine-grained Text Generation
Yun, L., Peng, L., & Shang, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.835
Demystifying optimized prompts in language models
Melamed, R., McCabe, L. H., & Huang, H. H. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.147
Integrating Vision and Language for Human Centered Interaction and RoboticsPreprint
Popescu, A., Marinou, S., Pereira, L., & Shyama, F. (2025) · doi:10.36227/techrxiv.176721030.05388084/v1
Mitigating Context-Memory Conflicts in LLMs Through Dynamic Cognitive Reconciliation Decoding
Zhou, Y., Li, W., Lu, Y., Wang, Y., Liu, X., Wang, W., Yu, J., Zhang, M., & Li, J. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3657980 · Cited by 2
Zhao, W. X., Zhou, K., Li, J., Tang, T., & Wen, J.-R. (2025) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-96-6259-3_3
Zhao, W. X., Zhou, K., Li, J., Tang, T., & Wen, J.-R. (2025) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-96-6259-3_4
S2Q: Teaching Language Models New Facts Through Knowledge Graph Instruction Synthesis
Zhao, Z., Cao, X., & Zhang, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-6196-4_17
Improving Retrieval-Augmented Generation for Medical Literature: A Case Study on Pertussis Research
Takabatake, H., Martono, N. P., Kuwae, A., Iuchi, T., & Ohwada, H. (2025) · International Conference on Information & Communication Technology and System (ICTS) · doi:10.1109/icts67612.2025.11369608
A Language Model Trained on Uruguayan Spanish News Text
Filevich, J. P., Marco, G., Castro, S., Chiruzzo, L., & Rosá, A. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4883cfyupepv
(2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3p5nrhhwdhbe
MossNet: Mixture of State-Space Experts is a Multi-Head Attention
Tuli, S., Smith, J. S., Jeelani, H., Lin, C.-H., Patel, A., Ramanishka, V., Hsu, Y.-C., & Jin, H. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.ijcnlp-long.26
BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
Jumelet, J., Fourtassi, A., Haga, A., Bunzeck, B., Shandilya, B., Galván-Sosa, D., Haznitrama, F. G., Padovani, F., Meyer, F., Hu, H., et al. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.152
Understanding Social Media Users' Perspectives for Content Reuse
Sinha, R., & Li, H. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3799454
Wang, Z., Li, Y., Fu, J., & Zhang, G. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_4
The Privacy Paradox of LLMs: User Perceptions and the Reality of PII Leakage
Cheng, S., Xu, H., Meng, S., Hao, S., Yue, C., & Li, Z. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791809
Scheuerman, M. K., Hutiri, W. T., Rahmattalabi, A., Matthews, V., Xiang, A., & Andrews, J. T. A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790364 · Cited by 1
Metadata
- Authors
- Soldaini, L., Kinney, R., Bhagia, A., Schwenk, D., Atkinson, D., Authur, R., Bogin, B., Chandu, K., Dumas, J., Elazar, Y., Hofmann, V., Jha, A., Kumar, S., Lucy, L., Lyu, X., Lambert, N., Magnusson, I., Morrison, J., Muennighoff, N., Naik, A., Nam, C., Peters, M., Ravichander, A., Richardson, K., Shen, Z., Strubell, E., Subramani, N., Tafjord, O., Walsh, E., Zettlemoyer, L., Smith, N., Hajishirzi, H., Beltagy, I., Groeneveld, D., Dodge, J., & Lo, K.
- Affiliations
- University of Washington (US)
- Allen Institute for Artificial Intelligence (US)
- Massachusetts Institute of Technology (US)
- ORCID
- Soldaini, Luca
- Kinney, Rodney
- Schwenk, Dustin
- Authur, Russell
- Hofmann, Valentin
- Jha, Ananya
- Kumar, Sachin
- Lucy, Li
- Lambert, Nathan
- Morrison, Jacob
- Muennighoff, Niklas
- Naik, Aakanksha
- Peters, Matthew
- Richardson, Kyle
- Shen, Zejiang
- Strubell, Emma
- Zettlemoyer, Luke
- Smith, Noah
- Hajishirzi, Hannaneh
- Groeneveld, Dirk
- Lo, Kyle
- Venue
- Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 15725-15788
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4402671286
- Cited by
- 56
- References
- 0
- Citations by year
- 2026: 22 · 2025: 23 · 2024: 11
- FWCI
- 16.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- open corpus, language model pretraining, multilingual corpora, low-resource languages, multilingual language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
