Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
In this paper, we propose data statements as a design solution and professional practice for natural language processing technologists, in both research and development. Through the adoption and widespread use of data statements, the field can begin to address critical scientific and ethical issues that result from the use of data from certain populations in the development of technology for other populations. We present a form that data statements can take and explore the implications of adopting them as part of regular practice. We argue that data statements will help alleviate issues related to exclusion and bias in language technology, lead to better precision in claims about how natural language processing research can generalize and thus better engineering results, protect companies from public embarrassment, and ultimately lead to language technology that meets its users in their own preferred linguistic style and furthermore does not misrepresent them to others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“or lead to adverse effects from deploying suboptimal or harmful models (Bender and Friedman, 2018) on tasks for which they are ill-suited (Raji et al., 2022).”
From Lessons from the Trenches on Reproducible Evaluation of Language Models (2024) · cites this paper as Bender and Friedman, 2018
The cited source says what the claim attributes to it.
In the citing paper
“Numerous authors have observed that ML systems perform better for users whose demographic characteristics match those represented in the training data [9,36].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · cited 1×
“Here, our focus is on linguistic data. the review (e.g., Pang et al., 2002) or the hashtag #sarcasm used to identify sarcastic language (e.g., Kreuz and Caucci, 2007).”
BackgroundThe Study of Second Language Acquisition
Brown, C., & Ellis, R. (1996) · The Modern Language Journal · doi:10.2307/329065 · Cited by 4,199
Publication Manual of the American Psychological Association
DeCleene, K. E., & Fogo, J. (2012) · Occupational Therapy In Health Care · doi:10.3109/07380577.2011.629024 · Cited by 3,167
The Social Stratification of English in New York City.
Crockett, H. J., & Labov, W. (1968) · American Sociological Review · doi:10.2307/2092899 · Cited by 3,084
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03 · doi:10.3115/1075096.1075150 · Cited by 3,008
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 4×
“Waseem and Hovy (2016) provide the following complete list of terms used in their initial scrape: MKR, asian drive, feminazi, immigrant, nigger, sjw, WomenAgainstFeminism, blameonenotall, islam terrorism, notallmen, victimcard, victim card, arab terror, gamergate, jsil, racecard, race card. dataset…”
BackgroundMethodInfluential
Cited by
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
“While documentation allows for potential accountability [13, 52, 86], undocumented training data perpetuates harm without recourse.”
BackgroundSelf-citationA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
“ces while using data, such as having datasheets that would act like a supporting document for the data reporting the dataset creation method, its characteristics, motivations, and its skews [12, 48]. [11] proposes a similar approach for the NLP applications. A similar suggestion has been proposed for models in [90]. A…”
MethodA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
“As another example, Bender and Friedman [2] outline a proposal similar to datasheets for datasets specifically intended for language-based datasets.”
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
“As another example, Bender and Friedman [2] outline a proposal similar to datasheets for datasets specifically intended for language-based datasets.”
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
“In recent years, novel data transformation approaches, which include data statements (Bender and Friedman 2018), datasheets, and nutrition labels, have been introduced.”
BackgroundAnil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
ChatGPT and Open-AI Models: A Preliminary Review
Roumeliotis, K. I., & Tselikas, N. D. (2023) · Future Internet · doi:10.3390/fi15060192 · Cited by 706
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643
Sex and gender analysis improves science and engineering
Tannenbaum, C., Ellis, R. P., Eyssel, F., Zou, J., & Schiebinger, L. (2019) · Nature · doi:10.1038/s41586-019-1657-6 · Cited by 581
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“Management Documentation [11, 22, 41, 129, 156, 246] Auditing [58, 228, 272, 277, 290, 292, 356] Cooperation [27, 90] Worklow MLOps [165, 233, 303]”
BackgroundAdvances, challenges and opportunities in creating data for trustworthy AI
Liang, W., Tadesse, G. A., Ho, D., Fei-Fei, L., Zaharia, M., Zhang, C., & Zou, J. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00516-1 · Cited by 559
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545
“defining contracts is the work that proposes standardized documentations to communicate the performance characteristics of trained AI models. The examples of such documentations are: data statements [6], datasheets for datasets [18], model cards [54], reproducibility checklists [56], fairness checklists [48], and fact…”
BackgroundFairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
“Other important aspects such as model/data transparency (Mitchell et al., 2019; Bender and Friedman, 2018) and privacy preservation (Reddy and Knight, 2016; Elazar and Goldberg, 2018; Li et al.”
BackgroundCrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434
Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z., & Çöltekin, Ç. (2020) · Proceedings of the Fourteenth Workshop on Semantic Evaluation · doi:10.18653/v1/2020.semeval-1.188 · Cited by 378
Datasets: A Community Library for Natural Language Processing
Lhoest, Q., Villanova del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., Tunstall, L., et al. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2021.emnlp-demo.21 · Cited by 343
“Datasets is backed by the Dataset Hub 4 that helps users navigate the growing number of available resources and draws inspiration from recent work calling for better documentation of ML datasets in general (Gebru et al., 2018) and NLP datasets in particular (Bender and Friedman, 2018).”
Directions in abusive language training data, a systematic review: Garbage in, garbage out
Vidgen, B., & Derczynski, L. (2020) · PLOS ONE · doi:10.1371/journal.pone.0243300 · Cited by 318
“To provide structure to how we analyse the datasets, we partly adopt the ‘data statements’ framework put forward by Bender and Friedman [48], as well as other work that provides frameworks, schema and processes for analysing NLP artefacts [49–51].”
BackgroundMethodInfluentialDocumenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306
“EN dataset. structured, task-specific NLP datasets, best practices have emerged around documenting the collection process, composition, intended uses, and other characteristics (Bender and Friedman, 2018; Gebru et al., 2018; Hutchinson et al., 2021).”
BackgroundDeduplicating Training Data Makes Language Models Better
Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.577 · Cited by 304
Release Strategies and the Social Impacts of Language Models
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1908.09203 · Cited by 292
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“So-called data statements [213] can provide developers and users with the context required to understand specific models’ potential biases.”
BackgroundShould ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282
“In (semi)supervised learning scenarios, where human annotators provide labels or annotations for the training data, biases may emerge from the subjective judgments of the annotators themselves, influencing the model’s understanding of the data [32, 33, 34].”
BackgroundInfluentialShould ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · arXiv · doi:10.48550/arxiv.2304.03738 · Cited by 282
AI Art and its Impact on Artists
Jiang, H. H., Brown, L., Cheng, J., Khan, M., Gupta, A., Workman, D., Hanna, A., Flowers, J., & Gebru, T. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604681 · Cited by 265
“While auditing, reporting, and transparency are well-known possible proposals for regulating AI in general [17, 22, 54, 83, 100], formulating sector and industry specific proposals is essential when it comes to effective governance [100], and is what will be needed for image generators and art.”
BackgroundExplainability fact sheets: a framework for systematic assessment of explainable approaches
Sokol, K., & Flach, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372870 · Cited by 264
Hutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
“right”. Indeed, premortems can be a useful tool during data requirements analysis for identifying gaps and vagueness. Transparent Dataset Reporting. Transparent reporting of machine learning datasets [16, 52, 70] and the models trained on those data [99] enables third party agents, including potential users, to make i…”
MethodFrom ethical AI frameworks to tools: a review of approaches
Prem, E. (2023) · AI and Ethics · doi:10.1007/s43681-023-00258-9 · Cited by 242
Sentiment Analysis for Fake News Detection
Alonso, M. A., Vilares, D., Gómez-Rodríguez, C., & Vilares, J. (2021) · Electronics · doi:10.3390/electronics10111348 · Cited by 239
“• It is known that NLP algorithms and resources may have inadvertently introduced biases [196,197].”
BackgroundNeural Unsupervised Domain Adaptation in NLP—A Survey
Ramponi, A., & Plank, B. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.603 · Cited by 230
Predictive biases in natural language processing models: A conceptual framework and overview
Shah, D. S., Schwartz, H. A., & Hovy, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.468 · Cited by 241
“Data State-ments Bender and Friedman (2018) suggest clearly disclosing data selection, annotation, and curation processes explicitly and trans-parently.”
BackgroundFive sources of bias in natural language processing
Hovy, D., & Prabhumoye, S. (2021) · Language and Linguistics Compass · doi:10.1111/lnc3.12432 · Cited by 298
“Bender and Friedman (2018) proposed a framework to document these decisions in a Data Statement.”
BackgroundData Cards: Purposeful and Transparent Dataset Documentation for Responsible AI
Pushkarna, M., Zaldivar, A., & Kjartansson, O. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533231 · Cited by 222
“In its original form, schema elements in Data Statements featured particular aspects of language datasets, including speech context, speaker demographic and annotator demographic—all of which were inspired by practices from the fields of psychology and medicine that require such disclosure about populations being stud…”
MethodComparison of text preprocessing methods
Chai, C. P. (2023) · Natural Language Engineering · doi:10.1017/s1351324922000213 · Cited by 212
“Here, we explicitly specify the language name to acknowledge the importance of data statement for NLP (Bender and Friedman 2018).”
MethodRe-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228
“, [34, 77, 136]), openly discussing limitations, failure cases, and lessons learnt can help move from the ‘magic pill’ role of fairness as a checklist for ethical issues in India—to a more pragmatic, flawed, and evolving scientific function.”
BackgroundBias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
Nasarian, E., Alizadehsani, R., Acharya, U. R., & Tsui, K.-L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102412 · Cited by 207
Hudley, A. H. C., Mallinson, C., & Bucholtz, M. (2020) · Language · doi:10.1353/lan.2020.0074 · Cited by 199
Is Chatgpt a menace for creative writing ability? An experiment
Niloy, A. C., Akter, S., Sultana, N., Sultana, J., & Rahman, S. I. U. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12929 · Cited by 198
Chan, B., Schweter, S., & Möller, T. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.598 · Cited by 187
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages
Nekoto, W., Marivate, V., Matsila, T., Fasubaa, T., Fagbohungbe, T., Akinola, S. O., Muhammad, S., Kabongo Kabenamualu, S., Osei, S., Sackey, F., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.195 · Cited by 182
HateCheck: Functional Tests for Hate Speech Detection Models
Röttger, P., Vidgen, B., Nguyen, D., Waseem, Z., Margetts, H., & Pierrehumbert, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.4 · Cited by 187
“Following Bender and Friedman (2018), we provide a data statement, which documents the generation and provenance of test cases in H ATE C HECK .”
MethodQuality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00447 · Cited by 179
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.12028 · Cited by 179
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection
Vidgen, B., Thrush, T., Waseem, Z., & Kiela, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.132 · Cited by 174
“Following Bender and Friedman (2018) we provide a data statement, which documents the process and provenance of the final dataset.”
MethodED-AI Lit: An Interdisciplinary Framework for AI Literacy in Education
Allen, L. K., & Kendeou, P. (2024) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/23727322231220339 · Cited by 168
“These perspectives underscore the critical nature of AI ethics education, guiding students to comprehend the ethical intricacies of AI technology and recognize the consequences of biased algorithms on diverse social groups (Bender & Friedman, 2018; Hutchinson & Mitchell, 2019).”
BackgroundDo Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development
Scheuerman, M. K., Hanna, A., & Denton, R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476058 · Cited by 216
“have all put forth data reporting frameworks for improving dataset documentation [10, 43, 57].”
BackgroundMethodInfluentialThe METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169
“While some initiatives cover the entire ML system and development pipeline (e.g., MINIMAR 39 , FactSheets 40 ), others are concerned with documentation surrounding the model (e.g., Model Cards 41 ), and still others concentrate on the documentation of datasets (e.g., Datasheets 42 , STANDING Together 43,44 , Dataset N…”
Background(2019) · doi:10.1145/3351095.3372862 · Cited by 167
Gender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 165
“Since this can lead to systematic errors and/or adverse social effects, bias investigation is not only a scientific and technical endeavor but also an ethical one, given the growing societal role of NLP applications (Bender and Friedman, 2018).”
BackgroundOperationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157
“For NLP, documentation should encompass the following areas: curation rationale, language variety, speaker demographic, annotator demographic, speech situation, text characteristics, and recording quality [110].”
BackgroundMethodInfluentialQA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
Rogers, A., Gardner, M., & Augenstein, I. (2023) · ACM Computing Surveys · doi:10.1145/3560260 · Cited by 156
“The field of AI ethics is currently working on documenting the speaker demographics and possible social biases [21, 88], with the idea that this would then be useful for model certification [178].”
BackgroundThe GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Aremu, A., Bosselut, A., Chandu, K. R., Clinciu, M.-A., Das, D., Dhole, K., et al. (2021) · Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) · doi:10.18653/v1/2021.gem-1.10 · Cited by 159
WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization
Ladhak, F., Durmus, E., Cardie, C., & McKeown, K. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.360 · Cited by 149
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
Miceli, M., Schuessler, M., & Yang, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415186 · Cited by 149
Utility is in the Eye of the User: A Critique of NLP Leaderboards
Ethayarajh, K., & Jurafsky, D. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.393 · Cited by 147
Understanding artificial intelligence ethics and safety
Leslie, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05684 · Cited by 134
Measuring algorithmically infused societies
Wagner, C., Strohmaier, M., Olteanu, A., Kıcıman, E., Contractor, N., & Eliassi-Rad, T. (2021) · Nature · doi:10.1038/s41586-021-03666-1 · Cited by 144
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445921 · Cited by 150
Z-Inspection ® : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134
Christou, P. (2023) · The Qualitative Report · doi:10.46743/2160-3715/2023.6406 · Cited by 125
XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process
Clement, T., Kemmerzell, N., Abdelaal, M., & Amberg, M. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010006 · Cited by 131
Bauer, E., Greisel, M., Kuznetsov, I., Berndt, M., Kollar, I., Dresel, M., Fischer, M. R., & Fischer, F. (2023) · British Journal of Educational Technology · doi:10.1111/bjet.13336 · Cited by 130
“Bias can be addressed by carefully documenting and curating NLP datasets and models (Bender & Friedman, 2018; Mohammad, 2022), applying specialized debiasing techniques (Utama et al., 2020), and choosing automation goals less vulnerable to known biases.”
BackgroundMethodFairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646
Howcroft, D. M., Belz, A., Clinciu, M.-A., Gkatzia, D., Hasan, S. A., Mahamood, S., Mille, S., van Miltenburg, E., Santhanam, S., & Rieser, V. (2020) · Proceedings of the 13th International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.23 · Cited by 127
“, 2020), aimed at making “critical information accessible that previously could only be found by users with great effort” (Bender and Friedman, 2018).”
BackgroundThe Importance of Modeling Social Factors of Language: Theory and Practice
Hovy, D., & Yang, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.49 · Cited by 122
“, the demographics and network information of speaker, receiver, and social relations, which requires us to collect and document our annotations (Bender and Friedman, 2018).”
BackgroundAI and the Everything in the Whole Wide World Benchmark
Raji, I. D., Bender, E. M., Paullada, A., Denton, E., & Hanna, A. (2021) · arXiv · doi:10.48550/arxiv.2111.15366 · Cited by 127
“…challenge of addressing this lack of context is proper documentation for these datasets, which is often underdeveloped [Gebru et al., 2020, Bender and Friedman, 2018] and the devaluation of the data work [Paullada et al., 2020, Sambasivan et al., 2021, Hutchinson et al., 2020, Jo and Gebru, 2020].”
BackgroundSelf-citationSocietal Biases in Language Generation: Progress and Challenges
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.330 · Cited by 131
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123
Integrating ChatGPT into EFL writing instruction: Benefits and challenges
Baskara, F. R. (2023) · International Journal of Education and Learning · doi:10.31763/ijele.v5i1.858 · Cited by 118
“In addition, if the training data does not represent different genders, ages, or other demographic factors, this can also introduce bias into the text generated by the technology [46].”
BackgroundMKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
Longpre, S., Lu, Y., & Daiber, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00433 · Cited by 280
Shah, C., & Bender, E. M. (2022) · ACM SIGIR Conference on Human Information Interaction and Retrieval · doi:10.1145/3498366.3505816 · Cited by 115
Self-citationConcealed Data Poisoning Attacks on NLP Models
Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115
“Given such biases, as well as the rapid entrance of NLP into high-stakes domains, it is key to develop methods for documenting and analyzing a dataset’s source, biases, and potential vulnerabilities, i.e., data provenance (Ge-bru et al., 2018; Bender and Friedman, 2018).”
MethodPeer review of GPT-4 technical report and systems card
Gallifant, J., Fiske, A., Levites Strekalova, Y. A., Osorio-Valencia, J. S., Parke, R., Mwavu, R., Martinez, N., Gichoya, J. W., Ghassemi, M., Demner-Fushman, D., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000417 · Cited by 111
Garrido-Muñoz, I., Montejo-Ráez, A., Martínez-Santiago, F., & Ureña-López, L. A. (2021) · Applied Sciences · doi:10.3390/app11073184 · Cited by 123
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective
Kiritchenko, S., Nejadgholi, I., & Fraser, K. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12590 · Cited by 172
“Further, data statements (Bender & Friedman, 2018) and datasheets for datasets (Gebru, Morgenstern, Vecchione, Vaughan, Wallach, Daumé, & Crawford, 2018) were designed to standardize the process of documenting datasets.”
MethodAlgorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 224
“Another qualitative framework is represented by Data statements (Bender and Friedman 2018), which is tailored for NLP, requiring domainspecific information on language variety and speaker demographics.Dataset Nutrition Labels (Holland et al. 2018) describe a complementary, quantitative framework,…”
BackgroundInfluentialToward a Perspectivist Turn in Ground Truthing for Predictive Computing
Cabitza, F., Campagner, A., & Basile, V. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25840 · Cited by 114
“Other relevant information such as those indicated in the previously mentioned Data Statements (Bender and Friedman 2018) or Datasheets for Datasets (Gebru et al. 2021), and regarding for example the data collection and sharing details or the intended uses for the collected data, should also be reported; • Collect add…”
BackgroundMethodInfluentialXie, J., Sun, L., & Zhao, Y. F. (2025) · Engineering · doi:10.1016/j.eng.2024.04.024 · Cited by 112
A survey of recent methods for addressing AI fairness and bias in biomedicine
Yang, Y., Lin, M., Zhao, H., Peng, Y., Huang, F., & Lu, Z. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104646 · Cited by 146
“81 We list the potential datasets that researchers can use to conduct fairness research in biomedical tasks in Table 2, and more datasets in the medical imaging domain can be found in the work of Xu et al. and Lara et al 3,19 .”
BackgroundPolicy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 160
“…models, nor are there any specific regulatory frameworks that enforce this practice at the moment of writing; however, pioneering work in this area argues that “drawing on values-sensitive practices can only bring about improvements in engineering and scientific outcomes” (Bender & Friedman, 2018).”
Offensive Language and Hate Speech Detection for Danish
Sigurbergsson, G. I., & Derczynski, L. (2019) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.1908.04531 · Cited by 198
“The resources and classifiers are available from the authors under CC-BY license, pending use in a shared task; a data statement [30] is included in the appendix.”
KLUE: Korean Language Understanding Evaluation
Park, S., Moon, J., Kim, S., Cho, W. I., Han, J., Park, J., Song, C., Kim, J., Song, Y., Oh, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09680 · Cited by 75
“We adopt the recently proposed documentation frameworks; datasheets [41] and data statements [6].”
MethodDemoting Racial Bias in Hate Speech Detection
Xia, M., Field, A., & Tsvetkov, Y. (2020) · International Workshop on Natural Language Processing for Social Media (SocialNLP) · doi:10.18653/v1/2020.socialnlp-1.2 · Cited by 149
COLD: A Benchmark for Chinese Offensive Language Detection
Deng, J., Zhou, J., Sun, H., Zheng, C., Mi, F., Meng, H. M. L., & Huang, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.796 · Cited by 226
The ethics of using generative AI for qualitative data analysis
Davison, R. M., Chughtai, H., Nielsen, P., Marabelli, M., Iannacci, F., Offenbeek, V. M., Tarafdar, M., Trenz, M., Techatassanasoontorn, A. A., Andrade, A. D., et al. (2024) · Information Systems Journal · doi:10.1111/isj.12504 · Cited by 175
“…or exacerbate the colonisation or marginalisation of other modes of knowledge, cultures, or values, by privileging a certain perspective on the data analysis process, for instance, one reflecting Western cultures, because of training data prevalently collected online (Bender & Friedman, 2018).”
What Will it Take to Fix Benchmarking in Natural Language Understanding?
Bowman, S. R., & Dahl, G. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.385 · Cited by 95
Geiger, R. S., Cope, D., Ip, J., Lotosh, M., Shah, A., Weng, J., & Tang, R. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00144 · Cited by 142
Exploring gender biases in ML and AI academic research through systematic literature review
Shrestha, S., & Das, S. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.976838 · Cited by 95
“By understanding the type and characteristics of the dataset, ML model creators and users will be able to gauge the prediction quality of the ML model and its appropriate application (Bender and Friedman, 2018).”
Background
Metadata
- Authors
- Bender, E. M., & Friedman, B.
- Affiliations
- University of Washington (US)
- Corresponding author
- Bender, Emily M.
- Friedman, Batya
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2018-12
- Volume
- 6
- Pages
- 587-604
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2911227954
- Semantic Scholar
- 97bfa89addc6e5d76361e4c1e296949cad887b86
- Cited by
- 854
- References
- 46
- Citations by year
- 2026: 108 · 2025: 96 · 2024: 102 · 2023: 154 · 2022: 127 · 2021: 177 · 2020: 68 · 2019: 20 · 2018: 1
- FWCI
- 132.62
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Science Foundation (1302709)
- University of Washington
- York University
- Macquarie University
- University of California, San Diego
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- natural language processing, algorithmic bias, underrepresented groups, ethical practice, linguistic diversity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.