The DOI matches its Crossref record; 67 of the 70 references it cites were found in a registry or an index.
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Open accessAbstract
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Typically trained on an enormous scale of uncurated Internet-based data, LLMs inherit stereotypes, misrepresentations, derogatory and exclusionary language, and other denigrating behaviors that disproportionately affect already-vulnerable and marginalized communities (Bender et al. 2021; Dodge et al. 2021; Sheng et al. 2021b).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Dodge et al. 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Typically trained on an enormous scale of uncurated Internet-based data, LLMs inherit stereotypes, misrepresentations, derogatory and exclusionary language, and other denigrating behaviors that disproportionately affect already-vulnerable and marginalized communities (Bender et al. 2021; Dodge et al. 2021; Sheng et al. 2021b).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Dodge et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Very recent research - most published within the last year - has begun to analyze dataset values [18, 40], question assumptions around dataset use [100], unpack what is represented in ML datasets [43, 88, 124] and establish basics of how an organized dataset lifecycle might proceed [74].”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [43]
The cited source says what the claim attributes to it.
In the citing paper
“LLM datasets are poorly documented with little understanding of their ultimate makeup62.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“The pre-release increase in specific terms could occur naturally and may already be reflected in the pretraining corpora of later LLMs, which were built from time-specific web snapshots (e.g., C4, April 2019 [31]).”
From Delving Into PubMed Records: How AI-Influenced Vocabulary has Transformed Medical Writing since ChatGPT (2025) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“Access to pre-training data enables leakage detection through overlap analysis, as highlighted in previous research (Dodge et al., 2021;Deng et al., 2023); even without direct access, details like data sources and time frames can reveal leakages (Li, 2023a;b).”
From Benchmarking Benchmark Leakage in Large Language Models (2024) · cites this paper as Dodge et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 4×
“This paradigm incentivizes the use of ever larger corpora (Kaplan et al., 2020; Henighan et al., 2020), with the biggest models now training on a substantial fraction of the publicly-available internet (Raffel et al.”
MethodInfluentialTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
Word association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,737
Research works printed but not linked: 7
- Isaac Caswell, Julia Kreutzer, Lisa Wang, Ahsan Wa- hab, D. V. Esch, Nasanbayar Ulzii-Orshikh, Al- lahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, S. Sarin, Sokhar Samb, B. Sagot, C. Rivera, Annette Rios Gonzales, Isabel Papadimitriou, S. Osei, Pedro Javier Ortiz Suárez, Iroro Orife, Kelechi Ogueji, Rubungo Andre Niyongabo, Toan Q. Nguyen, Math- ias Muller, A. Muller, S. Muhammad, N. Muham- mad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, M. Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, N. D. Silva, Sakine cCabuk Balli, Stella Rose Biderman, Alessia Battisti, A. Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayo- dele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, and Mofetoluwa Adeyemi. 2021. Quality at a glance: An audit of web-crawled multilingual datasets. In Proceedings of the AfricanNLP Workshop.
- Kenneth Church and Patrick Hanks. 1990. Word as- sociation norms, mutual information, and lexicogra- phy. Computational linguistics, 16(1):22-29.
- Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recogniz- ing textual entailment challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1-9, Prague. Association for Computational Linguistics.
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. Journal of Machine Learning Research.
- Jonathan Rosa. 2019. Looking like a language, sound- ing like a race. Oxford University Press. · DOI
- Rachael Tatman. 2020. What i won't build. WiNLP Workshop at ACL.
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In NeurIPS.
Cited by
Sorted by citations
Items 101–200 of 317
Reproducibility in NLP: What Have We Learned from the Checklist?
Magnusson, I., Smith, N. A., & Dodge, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.809 · Cited by 22
Self-citationJASMINE: Arabic GPT Models for Few-Shot Learning
Nagoudi, E. M. B., Abdul-Mageed, M., Elmadany, A., Inciarte, A. A., & Khondaker, M. T. I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1040 · Cited by 25
How user language affects conflict fatality estimates in ChatGPT
Steinert, C., & Kazenwadel, D. (2024) · Journal of Peace Research · doi:10.1177/00223433241279381 · Cited by 22
Extracting representative subset from extensive text data for training pre-trained language models
Suzuki, J., Zen, H., & Kazawa, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2022.103249 · Cited by 16
The ROOTS Search Tool: Data Transparency for LLMs
Piktus, A., Akiki, C., Villegas, P., Laurençon, H., Dupont, G., Luccioni, A. S., Jernite, Y., & Rogers, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.29 · Cited by 29
Ecosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14
Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models
Ma, Z., Sansom, J., Peng, R., & Chai, J. Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.72 · Cited by 44
Language machines: Toward a linguistic anthropology of large language models
Lamoureaux, S., Castelle, M., & Weichselbraun, A. (2025) · Journal of Linguistic Anthropology · doi:10.1111/jola.70033 · Cited by 13
Bias Detection for Customer Interaction Data: A Survey on Datasets, Methods, and Tools
Donald, A., Galanopoulos, A., Curry, E., Muñoz, E., Ullah, I., Waskow, M. A., Dabrowski, M., & Kalra, M. (2023) · IEEE Access · doi:10.1109/access.2023.3276757 · Cited by 12
Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp
Hong, R., Agnew, W., Kohno, T., & Morgenstern, J. (2024) · Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3689904.3694702 · Cited by 11
Self-citationA Survey on the Impact of Pre-Trained Language Models in Sentiment Classification Task
Gautam, H., Gaur, A., & Yadav, D. K. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00805-z · Cited by 14
Abrar, A., Oeshy, N. T., Kabir, M., & Ananiadou, S. (2025) · AI & Society · doi:10.1007/s00146-025-02721-z · Cited by 24
Risks of AI Foundation Models in EducationPreprint
Blodgett, S. L., & Madaio, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10024 · Cited by 39
Re-evaluating Theory of Mind evaluation in large language models
Hu, J., Sosa, F., & Ullman, T. (2025) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2023.0499 · Cited by 10
Automatic Histograms: Leveraging Language Models for Text Dataset Exploration
Reif, E., Qian, C., Wexler, J., & Kahng, M. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650798 · Cited by 10
GeoAuxNet: Towards Universal 3D Representation Learning for Multi-Sensor Point Clouds
Zhang, S., Xin, F., & Duan, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01892 · Cited by 10
Evaluating Synthetic Data Generation from User Generated Text
Chim, J., Ive, J., & Liakata, M. (2024) · Computational Linguistics · doi:10.1162/coli_a_00540 · Cited by 12
EfficientMoE: Optimizing Mixture-of-Experts Model Training With Adaptive Load Balance
Zeng, Y., Huang, C., Mei, Y., Zhang, L., Su, T., Ye, W., Shi, W., & Wang, S. (2025) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2025.3539297 · Cited by 15
A review of large language models and the recommendation task
Munson, J., Cuezze, T., Nesar, M. S. B., & Zosso, D. P. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00334-5 · Cited by 12
Intersectional Male-Centric and White-Centric Biases in Collective Concepts
Bailey, A. H., Williams, A., Poddar, A., & Cimpian, A. (2024) · Personality and Social Psychology Bulletin · doi:10.1177/01461672241232114 · Cited by 12
From silicon to solutions: AI's impending impact on research and discovery
Markowitz, D. M., Boyd, R. L., & Blackburn, K. G. (2024) · Frontiers in Social Psychology · doi:10.3389/frsps.2024.1392128 · Cited by 17
VAIV bio-discovery service using transformer model and retrieval augmented generation
Kim, S., & Yoon, J. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05903-6 · Cited by 9
A Review of the Challenges with Massive Web-Mined Corpora Used in Large Language Models Pre-training
Perełkiewicz, M., & Poświata, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81596-6_14 · Cited by 16
MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving
Lee, J., Park, J., Cha, S., Cho, J. H., & Sim, J. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756118 · Cited by 15
Training Text-to-Text Transformers with Privacy Guarantees
Ponomareva, N., Bastings, J., & Vassilvitskii, S. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.171 · Cited by 10
Li, Y., Guérin, F., & Lin, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29822 · Cited by 38
The lost data: how AI systems censor LGBTQ+ content in the name of safety
Chen, S. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00695-4 · Cited by 9
Low-Resourced Languages and Online Knowledge Repositories: A Need-Finding Study.
Nigatu, H. H., Canny, J., & Chasins, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642605 · Cited by 8
CNS-CLIP: Transforming a Neurosurgical Journal Into a Multimodal Medical Model
Alyakin, A., Kurland, D. B., Alber, D. A., Sangwon, K. L., Li, D., Tsirigos, A., Leuthardt, E. C., Kondziolka, D. S., & Oermann, E. K. (2024) · Neurosurgery · doi:10.1227/neu.0000000000003297 · Cited by 9
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Harvey, E., Kizilcec, R. F., & Koenecke, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732137 · Cited by 11
M2D2: A Massively Multi-Domain Language Modeling Dataset
Reid, M., Zhong, V., Gururangan, S., & Zettlemoyer, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.63 · Cited by 10
A Tool Kit for Relation Induction in Text Analysis
Stoltz, D. S., Taylor, M. A., & Dudley, J. S. K. (2024) · Sociological Methods & Research · doi:10.1177/00491241241233242 · Cited by 8
Biomedical Relation Extraction Using Dependency Graph and Decoder-Enhanced Transformer Model
Kim, S., Yoon, J., & Kwon, O. (2023) · Bioengineering · doi:10.3390/bioengineering10050586 · Cited by 8
Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval
Giorgi, J., Soldaini, L., Wang, B., Bader, G. D., Lo, K., Wang, L. L., & Cohan, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.549 · Cited by 11
AI language models could both help and harm equity in marine policymaking
Ziegler, M., Lothian, S. L., O’Neill, B., Anderson, R., & Ota, Y. (2025) · npj Ocean Sustainability · doi:10.1038/s44183-025-00132-7 · Cited by 14
Methodologies and their comparison in complex compound aspect-based sentiment analysis: A survey
Haznitrama, F. G., Choi, H., & Chung, C. (2025) · AI Open · doi:10.1016/j.aiopen.2025.02.002 · Cited by 9
A careful examination of large behavior models for multitask dexterous manipulation
Barreiros, J. A. L., Beaulieu, A., Bhat, A. D., Cory, R., Cousineau, E. A., Dai, H., Fang, C.-H., Hashimoto, K., Irshad, M. Z., Itkina, M., et al. (2026) · Science Robotics · doi:10.1126/scirobotics.aea6201 · Cited by 137
CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data
Turski, M., Stanisławek, T., Kaczmarek, K., Dyda, P., & Graliński, F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41682-8_22 · Cited by 12
Building Stereotype Repositories with Complementary Approaches for Scale and Depth
Dev, S., Jha, A., Goyal, J., Tewari, D., Dave, S., & Prabhakaran, V. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.9 · Cited by 8
The Rise and Design of Enterprise Large Language Models
O’Leary, D. E. (2024) · IEEE Intelligent Systems · doi:10.1109/mis.2023.3345591 · Cited by 7
Theme-Driven Keyphrase Extraction to Analyze Social Media Discourse
Romano, W., Sharif, O., Basak, M., Gatto, J., & Preum, S. M. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31391 · Cited by 7
Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer’s Disease
Chiumento, F., & Liu, M. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825149 · Cited by 7
Matsui, K. (2025) · Perspectives on Medical Education · doi:10.5334/pme.1929 · Cited by 6
“The pre-release increase in specific terms could occur naturally and may already be reflected in the pretraining corpora of later LLMs, which were built from time-specific web snapshots (e.g., C4, April 2019 [31]).”
Benchmarking Benchmark Leakage in Large Language Models
Xu, R., Wang, Z., Fan, R.-Z., & Liu, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18824 · Cited by 6
Data Similarity is Not Enough to Explain Language Model Performance
Yauney, G., Reif, E., & Mimno, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.695 · Cited by 13
Basoah, J., Chechelnitsky, D., Long, T., Reinecke, K., Zerva, C., Zhou, K., Díaz, M., & Sap, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732045 · Cited by 16
Self-citationData Leakage in Visual Datasets
Ramos, P., Ramos, R., & García, N. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00661 · Cited by 14
Security and Privacy of Large Language Models: Threat Taxonomy, Ethical Implications, and Governance
Pribisalić, M., & Martinčić-Ipšić, S. (2026) · AI · doi:10.3390/ai7050152 · Cited by 9
Kunz, J. (2024) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180754712 · Cited by 7
Language Models for Predicting Organic Synthesis Procedures
Vaškevičius, M., & Kapočiūtė-Dzikienė, J. (2024) · Applied Sciences · doi:10.3390/app142411526 · Cited by 6
From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?
Coil, A., & Shwartz, V. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.169 · Cited by 8
Noise-Robust De-Duplication at Scale
Silcock, E., D’Amico-Wong, L., Yang, J., & Dell, M. (2022) · National Bureau of Economic Research · doi:10.3386/w30726 · Cited by 7
Accelerating Retrieval Augmented Language Model via PIM and PNM Integration
Jang, J.-W., Oh, J., Kong, Y., Hong, J.-Y., Cho, S.-H., Lee, J., Yang, H., & Yang, J.-S. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756020 · Cited by 5
Transfer Learning for Code-Mixed Data: Do Pretraining Languages Matter?
Tatariya, K., Lent, H., & de Lhoneux, M. (2023) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2023.wassa-1.32 · Cited by 5
Liu, L., Zhaohui, X., He, Y., Wang, Y., Li, H., Li, X., & Han, Y. (2024) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3649329.3655986 · Cited by 6
Dietrich, J., & Kazzer, P. (2023) · Drug Safety · doi:10.1007/s40264-023-01322-3 · Cited by 5
Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge
Nishida, K., Yoshinaga, N., & Nishida, K. (2024) · Journal of Natural Language Processing · doi:10.5715/jnlp.31.407 · Cited by 6
Multilingual Sequence-to-Sequence Models for Hebrew NLP
Eyal, M., Noga, H., Aharoni, R., Szpektor, I., & Tsarfaty, R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.487 · Cited by 6
AI and Law: The Next GenerationPreprint
Lee, K., Cooper, A. F., Grimmelmann, J., & Ippolito, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4580739 · Cited by 14
NLPositionality: Characterizing Design Biases of Datasets and Models
Santy, S., Liang, J. T., Bras, R. L., Reinecke, K., & Sap, M. (2023) · arXiv · doi:10.48550/arxiv.2306.01943 · Cited by 31
“…populations in datasets and models: people who are non-binary (e.g., Dev et al., 2021), Black (e.g., Sap et al., 2019; Davidson et al., 2019), Latinx (e.g., Dodge et al., 2021), Native American (e.g., Mager et al., 2018); and people who are not native English speakers (e.g., Joshi et al., 2020).”
BackgroundEvaluating the Social Impact of Generative AI Systems
Solaiman, I., Talat, Z., Agnew, W. S., Ahmad, L., Baker, D., Blodgett, S. L., Chen, C., Daumé, H., Dodge, J., Duan, I., et al. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198940272.013.0025 · Cited by 7
Self-citationDigital twins are funhouse mirrors: Five systematic distortions
Peng, T., Brucks, M. S., Gui, G., Merlau, D. J., Fan, G. J., Sliman, M. B., Johnson, E. J., Althenayyan, A., Bellezza, S., Donati, D., et al. (2026) · Science Advances · doi:10.1126/sciadv.aeh8260 · Cited by 5
Efficient multi-task learning with instance selection for biomedical NLP
Bonfigli, A., Bacco, L., Pecchia, L., Merone, M., & Dell’Orletta⋄, F. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110050 · Cited by 4
The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating Large Language Models
Katzy, J., Popescu, R. M., Arie Van Deursen, A., & Izadi, M. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00025 · Cited by 4
Generative AI and Democratic Culture
Branford, J., Soulier, E., & Fichtner, L. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00953-x · Cited by 5
Xie, X., Wang, L., Xiao, L., Han, M., Liu, L., Xu, X., Wang, J., Song, Z. Q., & Liao, X. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756113 · Cited by 4
Xie, X., Wang, L., Xiao, L., Ruan, L., Zhang, T., Wang, J., Wang, Y., & Liao, X. (2025) · IEEE Transactions on Computers · doi:10.1109/tc.2025.3628629 · Cited by 3
Black-box data: a new paradigm for biomedicine in the AI era
Naef, L., & Bronstein, M. M. (2026) · Chemical Science · doi:10.1039/d6sc01189f · Cited by 4
A Comprehensive Evaluation of Tool-Assisted Generation Strategies
Jacovi, A., Caciularu, A., Herzig, J., Aharoni, R., Bohnet, B., & Geva, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.926 · Cited by 7
Dolomites : Domain-Specific Long-Form Methodical Tasks
Malaviya, C., Agrawal, P., Ganchev, K., Srinivasan, P., Huot, F., Berant, J., Yatskar, M., Das, D., Lapata, M., & Alberti, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00727 · Cited by 8
Sanders, J., Mobley, J., Miller, I., Sochacka, N. W., Jensen, P. A., & Jensen, K. (2025) · Journal of Engineering Education · doi:10.1002/jee.70037 · Cited by 5
State-of-the-Art in Language Technology and Language-centric Artificial Intelligence
Agerri, R., Agirre, E., Aldabe, I., Aranberri, N., Egurrola, J. M. A., Atutxa, A., Azkune, G., Campos, J. A., Casillas, A., Estarrona, A., et al. (2023) · Cognitive technologies · doi:10.1007/978-3-031-28819-7_2 · Cited by 4
Petrov, A. V., & Macdonald, C. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3699521 · Cited by 3
Quantifying Geospatial in the Common Crawl Corpus
Ilyankou, I., Wang, M., Cavazzi, S., & Haworth, J. (2024) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3678717.3691286 · Cited by 8
LLMs applied to web scraping and web crawling: a systematic review
Landeta-López, P., García, J. M., Guevara‐Vega, C., & Ruiz–Cortés, A. (2026) · Computing · doi:10.1007/s00607-026-01666-5 · Cited by 5
Preference Leakage: A Contamination Problem in LLM-as-a-judge
Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., & Liu, H. (2025) · arXiv · doi:10.48550/arxiv.2502.01534 · Cited by 134
“Golchin and Surdeanu [17] have developed “guided instruction” to flag contaminated instances.”
Fostering Linguistic Justice in Digital Spaces: Using Indigenous Methods in the Study of NLP
Gasparotto, M. (2024) · Emerging Media · doi:10.1177/27523543241305814 · Cited by 4
BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotations
Giovannini, S., Coppini, F., Gemelli, A., & Marinai, S. (2025) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-025-00563-5 · Cited by 7
Collaborative Growth: When Large Language Models Meet Sociolinguistics
Nguyen, D. (2025) · Language and Linguistics Compass · doi:10.1111/lnc3.70010 · Cited by 8
Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory
Hafner, F. S., Valdivia, A., & Rocher, L. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732112 · Cited by 7
CodeCleaner: Mitigating Data Contamination for LLM Benchmarking
Cao, J., Chen, S., Zhang, W., Lo, H. C., Li, Y., & Cheung, S.-C. (2025) · Proceedings of the 16th International Conference on Internetware · doi:10.1145/3755881.3755901 · Cited by 2
L3X: Long Object List Extraction from Long Documents
Singhania, S., Razniewski, S., & Weikum, G. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761460 · Cited by 2
Pütz, O. (2025) · Journal of Linguistic Anthropology · doi:10.1111/jola.70036 · Cited by 2
What do large language models represent?
Ruyant, Q. (2026) · Synthese · doi:10.1007/s11229-025-05416-6 · Cited by 5
Gasparotto, M. (2026) · Big Data & Society · doi:10.1177/20539517251406184 · Cited by 3
Who Invests, Who Gets Funded: Gender and Racial Bias in LLM-Generated Investment Advice
Wang, Y. E., & Gu, K. (2026) · Journal of Business Ethics · doi:10.1007/s10551-026-06251-6 · Cited by 2
You, J., Lee, J., Lee, S., & Kwon, H.-Y. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3777990 · Cited by 3
Data protection in the era of large language models: challenges and opportunities
Revesai, Z., & Chipfumbu, C. T. (2026) · AI and Ethics · doi:10.1007/s43681-026-01223-y · Cited by 5
Maroudas, S., Legkas, S., Malakasiotis, P., & Chalkidis, I. (2022) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2022.nllp-1.8 · Cited by 9
Bareeva, D., Dreyer, M., Pahde, F., Samek, W., & Lapuschkin, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00357 · Cited by 3
What Does It Mean to “Misuse” Research Data?
Pasquetto, I. V., Thomer, A., Acker, A., Chtena, N., & Desai, M. A. (2024) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1108 · Cited by 3
Principles for the responsible application of Generative AI
Clarke, R. (2025) · Computer law & security review · doi:10.1016/j.clsr.2025.106131 · Cited by 5
Almeida, T. S., Nogueira, R. F., & Pedrini, H. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5788 · Cited by 8
Ethical AI Use in IEP Development: A Guiding Framework
Coleman, O. F., & Waterfield, D. A. (2026) · Journal of Special Education Technology · doi:10.1177/01626434261419099 · Cited by 2
Lightweight Transformers for Conversational AI
Pressel, D., Liu, W., Johnston, M. O., & Chen, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-industry.25 · Cited by 2
Revisiting Queer Minorities in Lexicons
Ramesh, K., Kumar, S., & KhudaBukhsh, A. R. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.23 · Cited by 2
Class Cardinality Comparison as a Fermi Problem
Ghosh, S., Razniewski, S., & Weikum, G. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587334 · Cited by 2
Metadata
- Authors
- Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Published
- 2021
- Pages
- 1286-1305
- OpenAlex
- W3204096139
- Cited by
- 306
- References
- 65
- Citations by year
- 2026: 76 · 2025: 77 · 2024: 54 · 2023: 65 · 2022: 33 · 2021: 1
- FWCI
- 17.67
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Allen Institute for Artificial Intelligence
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- web crawling, corpus annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
