Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
In this paper, we propose data statements as a design solution and professional practice for natural language processing technologists, in both research and development. Through the adoption and widespread use of data statements, the field can begin to address critical scientific and ethical issues that result from the use of data from certain populations in the development of technology for other populations. We present a form that data statements can take and explore the implications of adopting them as part of regular practice. We argue that data statements will help alleviate issues related to exclusion and bias in language technology, lead to better precision in claims about how natural language processing research can generalize and thus better engineering results, protect companies from public embarrassment, and ultimately lead to language technology that meets its users in their own preferred linguistic style and furthermore does not misrepresent them to others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“or lead to adverse effects from deploying suboptimal or harmful models (Bender and Friedman, 2018) on tasks for which they are ill-suited (Raji et al., 2022).”
From Lessons from the Trenches on Reproducible Evaluation of Language Models (2024) · cites this paper as Bender and Friedman, 2018
The cited source says what the claim attributes to it.
In the citing paper
“Numerous authors have observed that ML systems perform better for users whose demographic characteristics match those represented in the training data [9,36].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · cited 1×
“Here, our focus is on linguistic data. the review (e.g., Pang et al., 2002) or the hashtag #sarcasm used to identify sarcastic language (e.g., Kreuz and Caucci, 2007).”
BackgroundThe Study of Second Language Acquisition
Brown, C., & Ellis, R. (1996) · The Modern Language Journal · doi:10.2307/329065 · Cited by 4,199
Publication Manual of the American Psychological Association
DeCleene, K. E., & Fogo, J. (2012) · Occupational Therapy In Health Care · doi:10.3109/07380577.2011.629024 · Cited by 3,167
The Social Stratification of English in New York City.
Crockett, H. J., & Labov, W. (1968) · American Sociological Review · doi:10.2307/2092899 · Cited by 3,084
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03 · doi:10.3115/1075096.1075150 · Cited by 3,008
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 4×
“Waseem and Hovy (2016) provide the following complete list of terms used in their initial scrape: MKR, asian drive, feminazi, immigrant, nigger, sjw, WomenAgainstFeminism, blameonenotall, islam terrorism, notallmen, victimcard, victim card, arab terror, gamergate, jsil, racecard, race card. dataset…”
BackgroundMethodInfluential
Cited by
Items 601–700 of 865
Udechukwu, L. M. (2026) · AI and Ethics · doi:10.1007/s43681-026-01069-4 · Cited by 2
Chen, C., Liao, M., Phadnis, A. A., Li, Y., High, A. C., Abdullah, S., & Sundar, S. S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791542 · Cited by 2
“This study focuses on harmful cases where computer systems systematically and unfairly discriminate against certain groups of individuals in favor of others [11].”
Data protection in the era of large language models: challenges and opportunities
Revesai, Z., & Chipfumbu, C. T. (2026) · AI and Ethics · doi:10.1007/s43681-026-01223-y · Cited by 5
“Complete retraining of the model from scratch without the data subject’s information costs between USD 50 and 100 million for frontier models and may still be insufficient if the individual’s information appears in multiple sources within the training corpus [63].”
Societal Frameworks Can Improve LLM Alignment
Stańczak, K., Meade, N., Bhatia, M., Zhou, H., Böttinger, K., Barnes, J., Stanley, J., Papernot, N., Chapados, N., Thérien, D., et al. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806486 · Cited by 10
“While the unwanted epistemic uncertainty can undermine the reliability of language models, certain types of uncertainty are not only unavoidable but essential for their ethical deployment [32].”
Analysis of Bias in NLP Models With Regression and Effect Sizes
Dayanık, E., Vu, N. T., & Padó, S. (2022) · Northern European Journal of Language Technology · doi:10.3384/nejlt.2000-1533.2022.3505 · Cited by 4
“Clearly, such biases have the potential to cause concrete harm for the disadvantaged groups or individuals (Bender and Friedman, 2018; Blodgett et al., 2020) and must be observed and controlled as far as possible.”
BackgroundInformation Extraction from Social Media: A Hands-on Tutorial on Tasks, Data, and Open Source Tools
Mishra, S., Rezapour, R., & Diesner, J. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557503 · Cited by 2
DiscoSense: Commonsense Reasoning with Discourse Connectives
Bhargava, P., & Ng, V. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.703 · Cited by 10
A Multidisciplinary Lens of Bias in Hate Speech
Lobo, P. R., Kwarteng, J., Russo, M., Fahimi, M., Scott, K. M., Ferrara, A., Sen, I., & Fernández, M. (2023) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3625007.3627491 · Cited by 8
“Ideally, careful consideration of the downstream task, the identification of potential biases that could arise at an early stage, and iterative value-sensitive documentation of datasets [26], could already undercut quality issues later down the pipeline.”
BackgroundGómez, F. J. D., & Galende, J. N. (2024) · Revista Tecnología Ciencia y Educación · doi:10.51302/tce.2024.19081 · Cited by 6
“…auditoría y evaluación de los modelos (Raji et al. , 2020), incluyendo, por ejemplo, la eva-luación del rendimiento a partir de métricas de equidad y fiabilidad, o el empleo de técni-cas de eliminación de sesgo algorítmico (Bender y Friedman, 2018; Dev y Phillips, 2019; Zhang et al. , 2018).”
BackgroundThe Researcher's Sidekick: Unleashing ChatGPT's Potential in Academic Research
Gupta, M., & Bhaskar, P. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1798-3.ch011 · Cited by 2
Machine Learning and the Analysis of Culture
Mützel, S., & Ollion, É. (2024) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197653609.013.39 · Cited by 6
Plant, R., Giuffrida, V., & Gkatzia, D. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101746 · Cited by 3
Essential work, invisible workers: The role of digital curation in COVID ‐19 Open Science
Pasquetto, I. V., Abdu, A. A., & Chtena, N. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24965 · Cited by 3
“…community — develop and adopt digital curation practices without referring to these practices as “ digital curation, ” nor linking and citing the long LIS tradition of digital curation (Bender & Friedman, 2018; Gebru et al., 2021; Mitchell et al., 2019; Paullada et al., 2021; Peng et al., 2021).”
Fausses promesses et stupidité organisationnelle : pourquoi l’IA n’est pas synonyme de performance
Desjardins, G., & Gould, A. M. (2024) · Ad machina l avenir de l humain au travail · doi:10.1522/radm.no8.1846 · Cited by 2
Understanding generative AI to harness its potentials and mini- mize risks: A perspective
Caselli, T. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.111951 · Cited by 2
Petri, J., Barbeira, P. B., Pesce, M., Xhardez, V., Laje, R., & Cotik, V. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104795 · Cited by 2
Accountable Data Analytics Start with Accountable Data: The LiQuID Metadata Model.
Oppold, S., & Herschel, M. (2020) · Cited by 7
Spindola, S., José, M. M., Oliveira, A. S., Cação, F. N., & Cozman, F. G. (2021) · doi:10.5753/eniac.2021.18302 · Cited by 2
“Similarly, [Ghaeini et al. 2020] uses the attention weights to interpret decisions from nat-ural language inference (NLI) models, and [Vig 2019a] found even evidence of model biases with the help of them, which is noteworthy for several current practical reasons [Bender and Friedman 2018].”
MethodAn Interactive Exploratory Tool for the Task of Hate Speech Detection
McMillan-Major, A., Paullada, A., & Jernite, Y. (2022) · Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP) · doi:10.18653/v1/2022.hcinlp-1.2 · Cited by 8
“Automated Automated content moderation can rely on analyses of the content itself using NLP or computer vision (CV), features of user dynamics, and hashing to match instances of pre-identified forbidden content.”
BackgroundInfluentialBinza, L., & Budree, A. (2022) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-19429-0_24 · Cited by 3
A Corpus for Outbreak Detection of Diseases Prevalent in Latin America
Dellanzo, A., Cotik, V., & Ochoa-Luna, J. E. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.44 · Cited by 2
“Finally, we present the data statements of our data (Bender and Friedman, 2018).”
BackgroundMethodUser-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool
Madnani, N., & Loukina, A. (2020) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/2020.nlposs-1.20 · Cited by 2
“(2019) or standardized data statements advocated by Bender and Friedman (2018).”
BackgroundMadureira, B. (2021) · Workshop on Teaching NLP (TeachingNLP) · doi:10.18653/v1/2021.teachingnlp-1.14 · Cited by 7
HeteroCorpus: A Corpus for Heteronormative Language Detection
Vásquez, J., Bel-Enguix, G., Andersen, S., & Ojeda-Trueba, S.-L. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.23 · Cited by 16
“We follow the guidelines specified by (Bender and Friedman, 2018) to produce a Long Form data statement.”
MethodOn the Dynamics of Gender Learning in Speech Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.12 · Cited by 13
“The last few years have witnessed a growing effort towards developing preventive measures (Bender and Friedman, 2018) and mitigating strategies”
BackgroundData Governance and Policy in Africa
Ndemo, B., Ndung’u, N., Odhiambo, S. A., & Shimeles, A. (2023) · Springer eBooks · doi:10.1007/978-3-031-24498-8 · Cited by 7
More Than Just a Policy: Day-to-Day Effects of Data Governance on the Data Scientist
Marivate, V. N. (2023) · Springer eBooks · doi:10.1007/978-3-031-24498-8_7 · Cited by 2
FactSheets for Hardware-Aware AI Models: A Case Study of Analog In Memory Computing AI Models
Dominique, B., Maghraoui, K. E., Piorkowski, D., & Herger, L. M. (2023) · IEEE International Conference on Software Services Engineering (SSE) · doi:10.1109/sse60056.2023.00029 · Cited by 5
“In a second paper from the NLP domain [15], though specifically made for NLP researchers, ’Data Statements’ are meant to ”address critical scientific and ethical issues that result from the use of data from certain populations in the development of technology for other populations.”
BackgroundDescribeML: A dataset description tool for machine learning
Giner-Miguelez, J., Gómez, A., & Cabot, J. (2023) · Science of Computer Programming · doi:10.1016/j.scico.2023.103030 · Cited by 2
Gangal, V., & Hovy, E. H. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.1 · Cited by 3
Issues in Computer Vision Data Collection: Bias, Consent, and Label Taxonomy
Dulhanty, C. (2020) · UWSpace (University of Waterloo) · Cited by 7
Responsible AI Challenges in End-to-end Machine LearningPreprint
Whang, S. E., Tae, K. H., Roh, Y., & Heo, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05967 · Cited by 8
Style is NOT a single variable: Case Studies for Cross-Style Language UnderstandingPreprint
Kang, D., & Hovy, E. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03663 · Cited by 2
Armengol-Estapé, J., Carrino, C. P., Rodriguez-Penagos, C. G., Bonet, O. D. G., Armentano-Oller, C., González-Agirre, A., Melero, M., & Villegas, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07903 · Cited by 3
“In general, we provide as much information as possible following Bender and Friedman (2018) guidelines when relevant.”
MethodProceedings of the Fifth Workshop on Teaching NLP
(2021) · doi:10.18653/v1/2021.teachingnlp-1 · Cited by 3
The 7th Workshop on Online Abuse and Harms (WOAH)
(2023) · doi:10.18653/v1/2023.woah-1 · Cited by 2
Letter to the Editor. Rise of large language models in neurosurgery.
Barrit, S., Hadwe, S. E., Carron, R., & Madsen, J. R. (2024) · Journal of Neurosurgery · doi:10.3171/2024.3.jns24610 · Cited by 7
Measuring complex psychological and sociological constructs in large-scale textPreprint
Herderich, A., Lasser, J., Galešić, M., Aroyehun, S. T., García, D., & Garland, J. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/tzc9p · Cited by 5
Role of Blockchain Technology in e‐HRM in the Era of Artificial Intelligence
Singh, A., Lakhera, G., Ojha, M., & Mishra, A. K. (2025) · Wiley eBooks · doi:10.1002/9781394314409.ch13 · Cited by 4
Now, It’s Personal : The Need for Personalized Word Sense Disambiguation
King, M., & Cook, P. F. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_079 · Cited by 1
Understanding Ethics in NLP Authoring and Reviewing
Benotti, L., Fort, K., Kan, M., & Tsvetkov, Y. (2023) · doi:10.18653/v1/2023.eacl-tutorials.4 · Cited by 6
Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks
Rodríguez, P., Azab, M., Silvert, B., Sanchez, R., Labson, L., Shah, H., & Moon, S. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.3 · Cited by 6
A survey of Polish ASR speech datasets
Junczyk, M. (2024) · Poznań Studies in Contemporary Linguistics · doi:10.1515/psicl-2023-0019 · Cited by 3
Alghannami, A. (2024) · Mağallaẗ al-Nūr li-l-dirāsāt al-insāniyyaẗ. · doi:10.69513/jnfh.v2.n4.en12 · Cited by 1
Rahman, M. R., Karim, R., Arefin, M. S., Dhar, P. K., Hossain, G., & Shimamura, T. (2025) · Discover Applied Sciences · doi:10.1007/s42452-024-06444-6 · Cited by 5
“Performing meticulous annotation is a very crucial and challenging task in the field of natural language processing [43].”
NLP Libraries, Energy Consumption and Runtime: An Empirical Study
Chattaraj, R., & Chimalakonda, S. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729396 · Cited by 3
“The quality of data significantly influences the performance of NLP systems [13, 47].”
Evaluating the Effects of Prompt Perturbation on Bias and Hallucination in Large Language Models
Yousefi, M., Shahi, A., Sharifi, M., Romera, A. J. J., Hoermann, S., & Piumsomboon, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6588-4_25 · Cited by 1
“Bender et al. [5] demonstrated that LLMs can perpetuate societal biases, leading to research on detection and mitigation strategies, such as reducing gender bias in word embeddings [8].”
Benchmarking digital labor against Fairwork principles: an (auto)ethnography of chatbot training
Tomičić, A. (2025) · AI & Society · doi:10.1007/s00146-025-02421-8 · Cited by 3
Reading Between the Lines: Exploring English Literature Through AI and Machine Learning
Sharma, R. (2025) · Journal of Informatics Education and Research · doi:10.52783/jier.v5i2.3123 · Cited by 2
Cultural data markets: Interpreting the popularity of public datasets
Rojas, A. A., & Twyman, M. D. (2025) · New Media & Society · doi:10.1177/14614448251359631 · Cited by 1
“However, these characteristics are often limited or inaccurate and undermine ethical and responsible dataset production (Bender and Friedman, 2018; Longpre et al., 2023: 2).”
Rethinking Intelligence Power and Epistemic Authority in the Age of Superhuman AI
Iseko, A. (2025) · International Journal of Science Technology and Society · doi:10.11648/j.ijsts.20251304.14 · Cited by 1
Ardanuy, M. C., Berganzo‐Besga, I., Huesca, R. S., & Cuadrada, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04624-6_20 · Cited by 3
Gender bias in Artificial Intelligence
Cuesta, J. (2025) · Journal of Economic Policy Reform · doi:10.1080/17487870.2025.2579039 · Cited by 5
“Friedman and Nissenbaum (1996) and Bender and Friedman (2018) expand this framework by differentiating between pre-existing biases from societal norms, technical biases from algorithmic design, and emergent biases from the misapplication of AI systems.”
Ayravainen, L. E. M., Hinds, J. M., & Davidson, B. I. (2025) · PLoS ONE · doi:10.1371/journal.pone.0336269 · Cited by 4
“The items included in the questionnaire were motivated by guidelines for dataset documentation [36].”
InfluentialDescriptor: Cultural Dialog and Safety Evaluation Multilingual Dataset (CAMEO8)
GUDIPATI, S. K., MISHRA, N. A., ANKAM, G., & PONUGOTI, S. (2025) · IEEE data descriptions. · doi:10.1109/ieeedata.2025.3638721 · Cited by 1
ChatCAS: A Multimodal Ceramic Multi-Agent Studio for Consultation, Image Analysis and Generation
Han, Y., Liu, D., Ren, Y., Lei, Z., Sun, L., & Li, J. (2025) · Electronics · doi:10.3390/electronics14183735 · Cited by 1
Funding and Procurement for Government AI Research and Innovation
Velsberg, O. (2025) · Springer eBooks · doi:10.1007/978-3-031-89853-2_6 · Cited by 2
Bias Amplification in AI Models Trained on Low-Resource Data: A Sociolinguistic Audit
Udeze, N. C. (2025) · International Journal of Scientific Research and Modern Technology. · doi:10.38124/ijsrmt.v4i12.1075 · Cited by 1
Wang, N., & Leem, S.-G. (2026) · IntechOpen eBooks · doi:10.5772/intechopen.1013232 · Cited by 4
Anik, A. I., & Bunt, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789087 · Cited by 2
“These calls gave rise to socio-technical approaches, where the focus of explanations includes factors outside of the algorithmic black-box [40, 44, 46, 88], and data-centric approaches, where the focus of the explanations is on the training dataset of the system [5, 13, 16, 17, 60, 84, 134].”
Kumar, A., Gupta, S., Yadav, R., Yadav, U. S., Saroj, S., & Kumar, V. (2026) · European Journal of Sustainable Development Research · doi:10.29333/ejosdr/18328 · Cited by 1
Triedman, H., Sarathy, J., Nanayakkara, P., Cummings, R., Kaptchuk, G., Kross, S., & Redmiles, E. M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791573 · Cited by 1
“Prior work in these areas have provided guidelines for documenting datasets [8, 37, 43] and AI and ML models [19, 52].”
Al-Azani, S., Abudalfa, S. I., & Samma, H. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104870 · Cited by 7
Li, C., Gülfidan, G., & Zhang-Kopf, Y. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100642 · Cited by 2
Zarabi, H. (2026) · Current Issues in Language Planning · doi:10.1080/14664208.2026.2728062 · Cited by 1
RETRACTED: A REVIEW ON NATURAL LANGUAGE PROCESSING Report: Statistics and Facts
Dinesh, P. M., Kumar, J. H., Sekar, S. C., Swarnambiga, M., & Ranjani, S. (2021) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1084/1/012037 · Cited by 2
“,[1] had proposed the technique for utilizing Data articulations In both innovative work advancements the information explanations as a plan arrangement and expert practice for normal language preparing is been utilized much of the time and in this paper that is examined quickly.”
MethodRetractedDube, S. (2021) · Springer eBooks · doi:10.1007/978-3-030-68624-6_16 · Cited by 1
Sequence or Pseudo-Sequence? An Analysis of Sequential Recommendation Datasets.
Woolridge, D., Wilner, S., & Glick, M. (2021) · ACM Conference on Recommender Systems (RecSys) · Cited by 12
Artificial Intelligence for the Built Environment
Tarabishy, S., Kosicki, M., & Tsigkari, M. (2021) · Structural integrity · doi:10.1007/978-3-030-82430-3_5 · Cited by 1
Clustering Examples in Multi-Dataset Benchmarks with Item Response Theory
Rodríguez, P., Htut, P. M., Lalor, J., & Sedoc, J. (2022) · Workshop on Insights from Negative Results in NLP (Insights) · doi:10.18653/v1/2022.insights-1.14 · Cited by 10
“1 Introduction Understanding and describing the data in natural language processing (NLP) benchmarks is crucial to ensuring their validity and reliability (Ferraro et al., 2015; Gebru et al., 2018; Bender and Friedman, 2018).”
BackgroundPolicy-focused Stance Detection in Parliamentary Debate Speeches
Abercrombie, G., & Batista-Navarro, R. (2022) · Northern European Journal of Language Technology · doi:10.3384/nejlt.2000-1533.2022.3454 · Cited by 13
“1 There, we also provide a full data statement, following the guidelines of Bender and Friedman (2018).”
MethodFeldman, S. S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-90802-3.00002-2 · Cited by 4
Liu, W. (2023) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-023-02012-0 · Cited by 1
Das, D., Ma, D., & Kang, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.243 · Cited by 7
Interpreting Indirect Answers to Yes-No Questions in Multiple Languages
Wang, Z., Hossain, M. D., Mathur, S., Melo, T. C. S., Özler, K. B., Park, K., Quintero, J., Rezaei, M., Shakya, S. N., Uddin, M., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.146 · Cited by 1
A Fine-Grained Taxonomy of Replies to Hate Speech
Yu, X., Zhao, A., Blanco, E., & Hong, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.450 · Cited by 4
Schoenherr, J. R., Chiou, E. K., & Goldshtein, M. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-15988-6.00003-0 · Cited by 1
Rajiv, B., Roshan, K., & Sankaradass, V. (2023) · International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI) · doi:10.1109/icdsaai59313.2023.10452490 · Cited by 3
“In [12], these determinants were implemented in the NetLogo simulation environment, with an objective of identifying the preference and competence gap of learners.”
MethodMeasuring Adversarial Datasets
Bai, Y., Huang, R., Viswanathan, V., Kuo, T.-S., & Wu, T. (2023) · doi:10.18653/v1/2023.artofsafety-1.4 · Cited by 2
Evaluating Stereotypical Biases and Implications for Fairness in Large Language Models
Cao, C., & Bandara, D. (2024) · doi:10.18260/1-2--45767 · Cited by 2
Liu, X., Rzepka, R., & Araki, K. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111883 · Cited by 1
Measuring Bias in Generated Text Using Language Models—GPT-2 and BERT
Masuma, F., Chakraborty, P., Islam, A.-A.-U., Talukder, P. C., Roy, P., & Yousuf, M. A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-0892-5_39 · Cited by 1
Experience: Bridging Data Measurement and Ethical Challenges with Extended Data Briefs
Rondina, M., Vetrò, A., Fabris, A., Silvello, G., Susto, G. A., Torchiano, M., & Martin, J. C. D. (2025) · Journal of Data and Information Quality · doi:10.1145/3726872 · Cited by 2
A standard reporting system for the environmental impact of machine learning
Nagi, K. (2025) · AI and Ethics · doi:10.1007/s43681-025-00738-0 · Cited by 1
“The development of the model cards framework in 2018 was inspired by Data Statements for Natural Language Processing, proposed by Bender and Friedman in [7] and Datasheets for Datasets, proposed by Gebru et al. [8, 9].”
Evaluating Neural Network-Based Arabic Machine Translation
Babaali, B. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0735-0.ch009 · Cited by 1
ALSA: Context-Sensitive Prompt Privacy Preservation in Large Language Models
Ma, H., Lü, W., Liang, Y., Wang, T., Zhang, Q., Zhu, Y., & Si, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736840 · Cited by 12
“The recent evolution of large language models (LLMs) has demonstrated astonishing capabilities across various natural language processing (NLP) tasks, such as question answering, summarization, and code generation [6, 29, 53].”
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Apollo (University of Cambridge) · doi:10.17863/cam.65493 · Cited by 1
Minssen, T., & Gerke, S. (2021) · Springer eBooks · doi:10.1007/978-3-662-63530-8_12 · Cited by 1
On Degrees of Freedom in Defining and Testing Natural Language Understanding
Sugawara, S., & Tsugita, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.861 · Cited by 3
Interpreting Answers to Yes-No Questions in User-Generated Content
Mathur, S., Park, K., Chinnappa, D., Kotamraju, S., & Blanco, E. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.942 · Cited by 1
XFORMAL: A Benchmark for Multilingual Formality Style TransferPreprint
Briakou, E., Lu, D., Zhang, K., & Tetreault, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04108 · Cited by 1
“We describe the process of collecting formal rewrites using data statements protocols (Bender and Friedman, 2018; Gebru et al., 2018).”
MethodA Discussion on Building Practical NLP Leaderboards: The Case of Machine TranslationPreprint
Santy, S., & Bhattacharya, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06292 · Cited by 6
“, 2019) as well as data sheets (Gebru et al., 2018; Bender and Friedman, 2018) which report specific metrics pertaining to the model.”
BackgroundContemporary Approaches for AI Governance in Financial InstitutionsPreprint
Preuss, B. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3773581 · Cited by 3
Campbell-Redl, R. (2023) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.22709743 · Cited by 1
Participant Flow Diagrams for Health Equity in AI ResearchPreprint
Ellen, J., de Matos, J. C. R. G., Viola, M., Gallifant, J., Quion, J. M., Celi, L. A., & Hussein, N. S. A. (2023) · medRxiv · doi:10.1101/2023.10.05.23296611 · Cited by 1
Spivak, E. (2026) · Research Square · doi:10.21203/rs.3.rs-11273530/v1 · Cited by 1
Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
Kroll, J. A. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445937 · Cited by 2
“These include data sheets [101], fact sheets [10], data statements [28], data nutrition labels [123], model cards [171], and other standardized”
BackgroundDisclosure or Marketing? Analyzing the Efficacy of Vendor Self-reports for Vetting Public-sector AI
Kuehnert, B., Johnson, N., Dotan, R., & Heidari, H. (2026) · Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3805689.3812220
“The importance of training data curation in shaping system behavior is well established in the FAccT literature [7, 30, 60].”
Dimensions of Transparency in NLP Applications.Preprint
Saxon, M., Levy, S., Wang, X., Albalak, A., & Wang, W. Y. (2021) · arXiv (Cornell University)
Metadata
- Authors
- Bender, E. M., & Friedman, B.
- Affiliations
- University of Washington (US)
- Corresponding author
- Bender, Emily M.
- Friedman, Batya
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2018-12
- Volume
- 6
- Pages
- 587-604
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2911227954
- Semantic Scholar
- 97bfa89addc6e5d76361e4c1e296949cad887b86
- Cited by
- 854
- References
- 46
- Citations by year
- 2026: 108 · 2025: 96 · 2024: 102 · 2023: 154 · 2022: 127 · 2021: 177 · 2020: 68 · 2019: 20 · 2018: 1
- FWCI
- 132.62
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Science Foundation (1302709)
- University of Washington
- York University
- Macquarie University
- University of California, San Diego
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- natural language processing, algorithmic bias, underrepresented groups, ethical practice, linguistic diversity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.