Datasheets for datasets
Open accessThe DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Abstract
Documentation to facilitate communication between dataset creators and consumers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Paradata needs may include the preservation of novel forms of information, such as datasheets explaining training datasets or model cards documenting an AI tool, and of these documents' relation to archival collections processed using them [ 26 ].”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [26]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Significant critiques of LLMs have been raised, including concerns about bias and fairness, misinformation, opaqueness due to a lack of interpretability and explainability, and environmental impact (Bender et al., 2021; Gebru et al., 2021; Strubell et al., 2019).”
From Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments (2023) · cites this paper as Gebru et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If ML driven algorithms are considered as epistemic resources that help us to improve understanding and/or make better predictions, then the statistical nature of ML de facto prevents groups underrepresented in the data from contributing to this epistemic resource on an equal ground to dominant groups (Gebru et al., 2021)”
From Algorithmic profiling as a source of hermeneutical injustice (2025) · cites this paper as Gebru et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The authors found that providing examples of what the requirements can look like helps to guide practitioners in clarifying their own rules.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“I show how archival theories have been animated in documenting data in the context of AI and ML, including through documentation frameworks like Datasheets for Datasets”
From Can AI provenance inform archival provenance? (2026) · cites this paper as Gebru et al. 2021
12 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,822
Minimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109
(2018) · Cited by 3,510
Self-citationA sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts
Pang, B., & Lee, L. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1218990 · Cited by 3,301
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Researchers at Google published follow-up work on model cards that document machine learning models [20] and released a data card (a lightweight version of a datasheet) along with the Open Images dataset [17].”
BackgroundSelf-citationMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 2×
“Of particular concern are recent examples showing that machine learning models can reproduce or amplify unwanted societal biases reflected in training data [4, 5, 9].”
BackgroundThe Effects of Automobile Safety Regulation
Peltzman, S. (1975) · Journal of Political Economy · doi:10.1086/260352 · Cited by 1,531
The Recent Past and Near Future of Risk and/or Need Assessment
Andrews, D. A., Bonta, J., & Wormith, J. S. (2006) · Crime & Delinquency · doi:10.1177/0011128705281756 · Cited by 1,507 · cited 1×
“Domains such as criminal justice (Garvie et al., 2016; Systems, 2017; Andrews et al., 2006), hiring and employment (Mann & O’Neil, 2016), critical infrastructure (O’Connor, 2017; Chui, 2017), and finance (Lin, 2012) all increasingly depend on machine learning methods.”
Background
Research works printed but not linked: 7
- Chui, G. Project will use AI to prevent or minimize electric grid failures, 2017.
- Garvie, C., Bedoya, A., and Frankle, J. The Perpetual Line-Up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, Washington, D.C., 2016.
- Holstein, K., Vaughan, J., Daumé, H, Dudík, M., and Wallach, H. Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of 2019 ACM CHI Conf. on Human Factors in Computing Systems.
- Krasin, I. et al. OpenImages: A public dataset for large-scale multi-label and multi-class image classification, 2017.
- O'Connor, M. How AI Could Smarten Up Our Water System, 2017.
- Doha Supply Systems. Facial Recognition, 2017.
- Yagcioglu, S., Erdem, A., Erdem, E., and Ikizler-Cinbis, N. RecipeQA: A challenge dataset for multimodal comprehension of cooking recipes. In Proceedings of the 2018 Conf. on Empirical Methods in Natural Language Processing.
Cited by
Items 201–300 of 1,039
Copyright in generative deep learning
Franceschelli, G., & Musolesi, M. (2022) · Data & Policy · doi:10.1017/dap.2022.10 · Cited by 97
Modernizing and designing evaluation frameworks for connected sensor technologies in medicine
Coravos, A., Doerr, M., Goldsack, J., Manta, C., Shervey, M., Woods, B., & Wood, W. A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0237-3 · Cited by 82
Song, D., Vold, A., Madan, K., & Schilder, F. (2021) · Information Systems · doi:10.1016/j.is.2021.101718 · Cited by 113
Naik, K., Goyal, R. K., Foschini, L., Chak, C. M., Thielscher, C., Zhu, H., Lu, J. J.-Q., Lehár, J., Pacanoswki, M. A., Terranova, N., et al. (2023) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3152 · Cited by 105
Challenges of responsible AI in practice: scoping review and recommended actions
Sadek, M., Kallina, E., Bohné, T., Mougenot, C., Calvo, R. A., & Cave, S. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-01880-9 · Cited by 79
“Gebru et al. (2021) proposed ‘data-sheets’ for data sets; a form of documentation that details the motivation for the creation and composition of a data set, its collection process and recommended use cases.”
A Survey of Human‐Centered Evaluations in Human‐Centered Machine Learning
Sperrle, F., El‐Assady, M., Guo, G., Borgo, R., Chau, D. H., Endert, A., & Keim, D. A. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14329 · Cited by 120
Telkamp, J. B., & Anderson, M. H. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05057-6 · Cited by 101
Machine Learning and Bias in Medical Imaging: Opportunities and Challenges
Vrudhula, A., Kwan, A. C., Ouyang, D., & Cheng, S. C. (2024) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.123.015495 · Cited by 104
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 112
“To begin with, transparency regarding the (composition of the) training and test datasets [250] as well as the process by which these are gathered and curated [112] has been recognized as crucially important.”
BackgroundMethodA causal perspective on dataset bias in machine learning for medical imaging
Jones, C., Castro, D. C., Ribeiro, F. D. S., Oktay, O., McCradden, M. D., & Glocker, B. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00797-8 · Cited by 151
“Meanwhile, we call for medical imaging datasets to be more transparent regarding the processes involved in collection and curation 45,72–76 —without such knowledge, we cannot make informed assumptions about the nature of dataset biases.”
On Releasing Annotator-Level Labels and Information in Datasets
Prabhakaran, V., Mostafazadeh Davani, A., & Diaz, M. (2021) · Proceedings of The Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop · doi:10.18653/v1/2021.law-1.14 · Cited by 75
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 77
“Efforts such as Datasheets for Datasets (Gebru et al., 2018) have played an important role in encouraging dataset transparency through articulating the intent of the dataset creators, summarizing the data collection processes, and warning downstream dataset users of potential biases in the data.”
BackgroundInfluentialHarnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 72
“Practitioner-facing tools (e.g., Microsoft’s HAX “ responsible AI ” toolkit, data sheets [24], model cards [46]) further boosted this approach’s real-world impact.”
BackgroundCan AI be racist? Color‐evasiveness in the application of machine learning to science assessments
Cheuk, T. (2021) · Science Education · doi:10.1002/sce.21671 · Cited by 72
Standards, frameworks, and legislation for artificial intelligence (AI) transparency
Lund, B. D., Orhan, Z., Mannuru, N. R., Bevara, R. V. K., Porter, B., Vinaih, M. K., & Bhaskara, P. (2025) · AI and Ethics · doi:10.1007/s43681-025-00661-4 · Cited by 137
“The impor - tance of these concepts lies in their ability to provide trust in the development of AI, to make informed decisions, and to encourage responsible behavior [ 24, 44].”
Thylstrup, N. B. (2022) · Media Culture & Society · doi:10.1177/01634437211060226 · Cited by 97
“…to outline why and how critical archival theory allows us to unpack not only the inequities and injustices of data set assemblages (Andrew et al., 2020; Gebru et al., 2021; Pipkin, 2020; Stevens and Keyes, 2021), out but also the deeper politics of archival erasures and remains in machine cultures.”
BackgroundInfluentialREVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Narayanan, A., & Russakovsky, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_43 · Cited by 98
Zhang, A., Walker, O., Nguyen, K., Dai, J., Chen, A., & Lee, M. K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579601 · Cited by 72
“…and practitioners when creating fair AI systems in order to inform techniques that can help them [44, 69] such as AI fairness checklists [70], provenance for datasets [37, 46], visualizations of model behaviors [103], and toolkits for detecting and mitigating against ML unfairness [8, 10, 89].”
BackgroundMineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Fan, L., Wang, G., Jiang, Y., Mandlekar, A., Yang, Y., Zhu, H., Tang, A., Huang, D.-A., Zhu, Y., & Anandkumar, A. (2022) · arXiv · doi:10.48550/arxiv.2206.08853 · Cited by 72
“We present a Datasheet [39] for documentation and responsible usage of our internet knowledge databases.”
MethodBeyond explainability: justifiability and contestability of algorithmic decision systems
Henin, C., & Le Métayer, D. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01251-8 · Cited by 71
“In order to facilitate communication and to ensure comprehensiveness, the information could be provided in standardized formats such as Datasheets for Data Sets [34] and Model Cards [57]. are complete only if they establish a continuous link between the high-level objectives of the ADS (the applicable norms, for examp…”
How We Do Things With Words: Analyzing Text as Social and Cultural Data
Nguyen, D., Liakata, M., DeDeo, S., Eisenstein, J., Mimno, D., Tromble, R., & Winters, J. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00062 · Cited by 165
“These questions complement recent work providing guidance and suggestions about datasets (Bender and Friedman, 2018; Gebru et al., 2018) and models (Mitchell et al., 2019) more specifically.”
Shen, H., Deng, W. H., Chattopadhyay, A., Wu, Z. S., Wang, X., & Zhu, H. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445971 · Cited by 102
“Datasheets [28] focuses more on clarifying the characteristics of the data feeding into the ML model.”
BackgroundMa, Z., Mei, Y., Long, Y., Su, Z., & Gajos, K. Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642482 · Cited by 127
“LGBTQ+ participants’experienceinusingthechatbotsshowsthatthegenericpurposessofLLMstrainedonlargecorpusmightnotbeinclusive—howthedataiscollected ,annotated,andused,aswellaswhois involvedinthecurationanddesigningprocessescanhavesignif-icantimplicationsforLGBTQ+users[40].”
A Large-scale Synthetic Pathological Dataset for Deep Learning-enabled Segmentation of Breast Cancer
Ding, K., Zhou, M., Wang, H., Gevaert, O., Metaxas, D., & Zhang, S. (2023) · Scientific Data · doi:10.1038/s41597-023-02125-y · Cited by 99
“• Datasheets for Datasets To help users understand details of the dataset and possible extensive use, we prepared datasheets 44 for our data set.”
MethodThe poverty of ethical AI: impact sourcing and AI supply chains
Muldoon, J., Cant, C., Graham, M., & Ustek Spilda, F. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01824-9 · Cited by 88
Whose AI Dream? In search of the aspiration in data annotation.
Wang, D., Prabhat, S., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502121 · Cited by 70
“At a practice level, as the push for data documentation is picked up by ML practitioners, we too see this as an opportunity where the data labor practice can be documented and reviewed [25, 38, 39].”
BackgroundPalladino, N. (2022) · Telecommunications Policy · doi:10.1016/j.telpol.2022.102479 · Cited by 106
Application and Construction of Deep Learning Networks in Medical Imaging
Torres-Velázquez, M., Chen, W., Li, X., & McMillan, A. B. (2020) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2020.3030611 · Cited by 83
“In datasheets for datasets presented by Gebru et al. [167], a list of questions can be asked to enhance reliability, including motivations, composition, collection process, preprocessing, uses, distribution, and maintenance.”
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, E. (2020) · arXiv · doi:10.48550/arxiv.2001.00964 · Cited by 69
Self-citatione-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks
Kayser, M., Camburu, O.-M., Salewski, L., Emde, C., Do, V., Akata, Z., & Lukasiewicz, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00128 · Cited by 68
Data Preparation for Software Vulnerability Prediction: A Systematic Literature Review
Croft, R., Xie, Y., & Babar, M. A. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3171202 · Cited by 96
“Gebru and colleagues [73] propose a process called Datasheets for Datasets.”
BackgroundToward Explainable Affective Computing: A Review
Cortiñas-Lorenzo, K., & Lacey, G. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3270027 · Cited by 111
“In addition, standardization of dataset metrics and documentation, potentially adapting frameworks, such as Datasheets for Datasets [71], could help practitioners and researchers better understand the behavior of their developed systems in light of the data being used.”
BackgroundInfluentialInitial policy considerations for generative artificial intelligence
(2023) · OECD Artificial Intelligence Papers · doi:10.1787/fae2d1e6-en · Cited by 36
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 123
Ethics and society review: Ethics reflection as a precondition to research funding
Bernstein, M. S., Levi, M., Magnus, D. C., Rajala, B., Satz, D. M., & Waeiss, Q. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2117261118 · Cited by 77
“Among the challenges are oversights in who is and is not represented in the dataset (33), who has a seat at the table in the design and deployment of the AI (34), who is intended to benefit and be harmed by the AI (35), and what likely consequences might arise (36, 37).”
BackgroundA Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability
Sloane, M., Moss, E., & Chowdhury, R. (2022) · Patterns · doi:10.1016/j.patter.2021.100425 · Cited by 65
Building Trustworthy AI Solutions: A Case for Practical Solutions for Small Businesses
Crockett, K. A., Colyer, E., Gerber, L., & Latham, A. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3137091 · Cited by 96
“The workshops introduced a variety of ethical toolkits and activities with SMEs including datasheets for datasets [42], consequence scanning [43], conducting a data privacy impact assessment [44], and examining the risk to stakeholders of an AI recruitment tool using padlet [45].”
BackgroundSingh, H., Mhasawade, V., & Chunara, R. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000023 · Cited by 102
“Following guidelines developed for documenting datasets (72) and models (73) in the machine learning community, similar guidelines should be established for models in healthcare as well (10).”
MethodEvaluation Gaps in Machine Learning Practice
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533233 · Cited by 62
“We echo calls for more reflexivity around social and intentional factors around model development [116], more documentation of the complete lifecycle of model development [82, 167], and greater transparency around ML models and their datasets [13, 62, 117].”
Stalnaker, T., Wintersgill, N., Chaparro, O., Penta, M. D., Germán, D. M., & Poshyvanyk, D. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623347 · Cited by 128
“There are different proposals to track datasets and AI model information [43, 66, 72, 96].”
BackgroundHard choices in artificial intelligence
Dobbe, R., Krendl Gilbert, T., & Mintz, Y. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103555 · Cited by 65
Artificial intelligence bias in medical system designs: a systematic review
Kumar, A., Aelgani, V., Vohra, R., Gupta, S. K., Bhagawati, M., Paul, S., Saba, L., Suri, N., Khanna, N. N., Laird, J. R., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16029-x · Cited by 65
Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603203 · Cited by 65
Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words
Zhou, K., Ethayarajh, K., Card, D., & Jurafsky, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.45 · Cited by 99
Principles to Practices for Responsible AI: Closing the Gap
Schiff, D., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2020) · arXiv · doi:10.48550/arxiv.2006.04707
“Narrower tools and methodologies such as bias mitigation, privacy-by-design, and product design documentation [27, 40] can then be subsumed under this more comprehensive framework.”
MethodMedical artificial intelligence for clinicians: the lost cognitive perspective
Tikhomirov, L., Semmler, C., McCradden, M. D., Searston, R. A., Ghassemi, M., & Oakden‐Rayner, L. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00095-5 · Cited by 84
Schramowski, P., Tauchmann, C., & Kersting, K. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533192 · Cited by 217
“Here we ask the arguably “circular” question of whether a machine can help us reflect on inappropriate content, answering Question 16 in Datasheets [Gebru et al. 2021].”
BackgroundMethodInfluentialTrust criteria for artificial intelligence in health: normative and epistemic considerations
Kostick, K. M., Lang, B. H., Smith, J. N., Hurley, M. E., & Blumenthal‐Barby, J. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2023-109338 · Cited by 75
“Similar to how Gebru et al 33 argue that datasheets describing the creation and use of data sets can help to inform decisions about using a data set, we argue that providing relevant information about an AI tool’s development can help end-users appropriately gauge how much to trust (ie, rely on) a tool.”
BackgroundArtificial Intelligence and Deep Learning in Neuroradiology: Exploring the New Frontier
Kaka, H., Zhang, E., & Khan, N. M. (2020) · Canadian Association of Radiologists Journal · doi:10.1177/0846537120954293 · Cited by 70
Visual Semantic Role Labeling for Video Understanding
Sadhu, A., Gupta, T., Yatskar, M., Nevatia, R., & Kembhavi, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00554 · Cited by 117
Groh, M., Harris, C., Daneshjou, R., Badri, O., & Koochek, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555634 · Cited by 101
Artifical Intelligence and Bias: Challenges, Implications, and Remedies
Min, A. (2023) · Journal Of Social Research · doi:10.55324/josr.v2i11.1477 · Cited by 104
Dell, M. (2025) · Journal of Economic Literature · doi:10.1257/jel.20241733 · Cited by 107
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548
“For example, a glaring under-representation of darker skinned subjects has been identified within prominent facial analysis datasets [Buolamwini and Gebru, 2018] and the images in object recognition datasets have been overwhelmingly sourced from Western countries [DeVries et al., 2019].”
BackgroundYou reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 59
“Such issues are in part ex-186 acerbated by the general poverty of documentation of datasets (Gebru et al., 2018; Bender and Friedman, 2018) and machine learning models (Mitchell et al., 2019).”
BackgroundTROMPA-MER: an open dataset for personalized music emotion recognition
Gómez-Cañón, J. S., Gutiérrez-Páez, N. F., Porcaro, L., Porter, A., Cano, E., Herrera, P., Gkiokas, A., Santos, P., Hernández‐Leo, D., Karreman, C., et al. (2022) · Journal of Intelligent Information Systems · doi:10.1007/s10844-022-00746-0 · Cited by 72
“Following recommendations from Gebru et al. [44] and Prabhakaran et al. [45] to document the properties a dataset, we offer an interactive ”datasheet” for our dataset: a complementary website to listen to the music collection, emotion representations, and participants’ annotations and agreement.”
MethodKawakami, A., Chowdhary, S., Iqbal, S. T., Liao, Q. V., Olteanu, A., Suh, J., & Saha, K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610207 · Cited by 100
Heger, A. K., Marquis, L. B., Vorvoreanu, M., Wallach, H., & Wortman Vaughan, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555760 · Cited by 58
Self-citationTowards an AI-Enhanced Cyber Threat Intelligence Processing Pipeline
Alevizos, L., & Dekker, M. (2024) · Electronics · doi:10.3390/electronics13112021 · Cited by 105
Reducing AI bias in recruitment and selection: an integrative grounded approach
Soleimani, M., Intezari, A. H., Arrowsmith, J., Pauleen, D. J., & Taşkın, N. (2025) · The International Journal of Human Resource Management · doi:10.1080/09585192.2025.2480617 · Cited by 123
“This strategy helps in reducing the risk of perpetuating biases by increasing the representativeness of the training dataset (Gebru et al., 2021).”
Fairness in Deep Learning: A Survey on Vision and Language Research
Parraga, O., Móre, M. D., Mattjie, C., Gavenski, N. S., Kupssinskü, L. S., Medronha, A., de Moura, L. V., Simões, G. S., & Barros, R. C. (2023) · ACM Computing Surveys · doi:10.1145/3637549 · Cited by 123
“Datasheets for datasets [62] raises several questions (grouped into sections that roughly match the key stages of a dataset life cycle) to help dataset creators and consumers think about a dataset holistically, asking details on the motivation for creating the dataset, how the data was collected…”
Lekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58
“Datasheets for datasets, is a standardisation method proposed by Gebru et al [48] for documenting, among others, every dataset’s motivation, composition, collection process, and recommended uses.”
MethodFacilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 88
“One possible direction is to leverage proposed documentation for data [28] or for models [12, 53].”
BackgroundCommunity-in-the-loop: towards pluralistic value creation in AI, or—why AI needs business ethics
Häußermann, J. J., & Lütge, C. (2022) · AI and Ethics · doi:10.1007/s43681-021-00047-2 · Cited by 57
Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 103
“The standardized documentation for data (i.e., Data Statements (Bender & Friedman, 2018), Datasheets (Gebru et al., 2021), and Data Nutrition Labels (Holland et al., 2020; Chmielinski et al., 2022) and AI models (i.e., Model Cards (Mitchell et al., 2019), Interactive Model Cards (Crisan et al.,…”
InfluentialDynaSent: A Dynamic Benchmark for Sentiment Analysis
Potts, C., Wu, Z., Geiger, A., & Kiela, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.186 · Cited by 56
“(Gebru et al., 2018) that describes the data collection process and its motivations, and seeks to articulate known limitations of the resource.”
BackgroundInfluentialSoftware Architecture Challenges for ML Systems
Lewis, G. A., Özkaya, İ., & Xu, X. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00071 · Cited by 57
Fair ranking: a critical review, challenges, and future directions
Patro, G. K., Porcaro, L., Mitchell, L., Zhang, Q., Zehlike, M., & Garg, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533238 · Cited by 110
Feature Guided Masked Autoencoder for Self-Supervised Learning in Remote Sensing
Wang, Y., Hernández, H. H., Albrecht, C. M., & Zhu, X. X. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3493237 · Cited by 101
Teach Me to Explain: A Review of Datasets for Explainable NLP.Preprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · Cited by 64
Li, Y., & Goel, S. (2025) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2025.100739 · Cited by 121
Behavioral Use Licensing for Responsible AI
Contractor, D., McDuff, D., Haines, J. K., Lee, J., Hines, C., Hecht, B., Vincent, N., & Li, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533143 · Cited by 54
“TheDatasheets [29] for Datasets proposal introduces a questionnaire that captures the details about the purpose and nature of a system’s data choices, including details around labeling, sources of noise, source of the dataset, who the creators are, and whether there might be relationships between data samples (e.”
BackgroundAI Documentation: A path to accountability
Königstorfer, F., & Thalmann, S. (2022) · Journal of Responsible Technology · doi:10.1016/j.jrt.2022.100043 · Cited by 55
Zheng, C., Wang, D., Wang, A. Y., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517615 · Cited by 52
Kawakami, A., Coston, A., Zhu, H., Heidari, H., & Holstein, K. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642849 · Cited by 54
“These toolk-its are intended to support and document assessments of AI systems, including their (potential) impacts (e.g., [52]), intended use cases (e.g., [39]), capabilities and limitations (e.g., [53]), dataset quality (e.g., [19, 49]), and performance measures (e.g., [39]).”
BackgroundRisk of Training Diagnostic Algorithms on Data with Demographic Bias
Abbasi-Sureshjani, S., Raumanns, R., Michels, B. E. J., Schouten, G., & Cheplygina, V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61166-8_20 · Cited by 71
Modeling community standards for metadata as templates makes data FAIR
Musen, M. A., O’Connor, M. J., Schultes, E., Martínez-Romero, M., Hardi, J., & Graybeal, J. B. (2022) · Scientific Data · doi:10.1038/s41597-022-01815-3 · Cited by 70
“Workers at Microsoft Research have proposed the idea that datasets should be accompanied by a “datasheet” [20] that enumerates essential information about the data—metadata that have aspects similar to those of a minimal information checklist.”
Assessing and addressing algorithmic bias in practice
Cramer, H. S. M., Garcia-Gathright, J. I., Springer, A., & Reddy, S. (2018) · interactions · doi:10.1145/3278156 · Cited by 63
Teach Me to Explain: A Review of Datasets for Explainable Natural Language ProcessingPreprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12060 · Cited by 149
“It is important to constantly evaluate data collection practices critically and standardize them [13, 39, 95].”
BackgroundMethodChanging the World by Changing the Data
Rogers, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.170 · Cited by 51
“(2021), is only that whatever categories and demographics went into the data design, they have to be documented (Bender and Friedman, 2018; Gebru et al., 2020) and made explicit, so that the users could be informed about what”
BackgroundInfluentialImplementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble
Ali, W. B., Pesaranghader, A., Avram, R., Overtchouk, P., Perrin, N., Laffite, S., Cartier, R., Ibrahim, R., Modine, T., & Hussin, J. (2021) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.711401 · Cited by 61
“This is particularly important in raw health data that generally lack maintenance and validation and raise important interoperability problems (113, 114).”
BackgroundEl–Sappagh, S., Alonso, J. M., Abuhmed, T., Ali, F., & Bugarín, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10415-5 · Cited by 69
Transparency of medical artificial intelligence systems
Kim, C., Gadgil, S. U., & Lee, S. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00363-w · Cited by 81
Representativeness in Statistics, Politics, and Machine Learning
Chasalow, K., & Levy, K. E. C. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445872 · Cited by 89
Fair Generative Modeling via Weak SupervisionPreprint
Choi, K., Grover, A., Singh, T., Shu, R., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12008 · Cited by 124
“, survey data collected by organizations such as World Bank typically follow several good dataset collection practices [9] to ensure representativeness before release.”
BackgroundZeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
Cabrera, Á. A., Fu, E., Bertucci, D., Holstein, K., Talwalkar, A., Hong, J. I., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581268 · Cited by 51
Intrinsic Evaluation of Summarization Datasets
Bommasani, R., & Cardie, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.649 · Cited by 95
“Data understanding can also inform scientific and ethical decision-making (Bender and Friedman, 2018; Gebru et al., 2018; Mitchell et al., 2019) with recent work studying how social biases encoded in training data propagate to learned models (Zhao et al., 2019; Tan and Celis, 2019).”
BackgroundLa Fors, K., Custers, B., & Keymolen, E. (2019) · Ethics and Information Technology · doi:10.1007/s10676-019-09503-4 · Cited by 50
“Searching for application-based approaches yielded two lists, i.e., Google’s AI principles aimed at ethical principles for daily use and the datasheets for datasets approach (Gebru et al. 2018).”
MethodBad Seeds: Evaluating Lexical Methods for Bias Measurement
Antoniak, M., & Mimno, D. (2021) · doi:10.18653/v1/2021.acl-long.148 · Cited by 95
An Ethical Framework for Guiding the Development of Affectively-Aware Artificial Intelligence
Ong, D. C. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597441 · Cited by 66
Towards Training Reproducible Deep Learning Models
Chen, B.-Y., Wen, M., Shi, Y., Lin, D., Rajbahadur, G. K., & Jiang, Z. (2022) · International Conference on Software Engineering · doi:10.1145/3510003.3510163 · Cited by 66
“Various documentation frameworks for DL models [30, 53] or checklists [19] have been proposed recently.”
BackgroundChristoforaki, M., & Beyan, O. (2022) · Applied Sciences · doi:10.3390/app12094130 · Cited by 74
“…to hold sensitive data, identify data bias beforehand, or act as a pre-processor and giving access accordingly, e.g., by enforcing anonymization [83]; and the use of mechanisms such as data trusts, i.e., legal structures that provide mechanisms “for individuals to pool their data rights (or…”
BackgroundA hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
“Providing potential ways forward, researchers have proposed documentation artefacts ( e.g., Datasheets for Datasets [30], Data Cards [77], Data Statements [5]) and archival artefacts of the decisions made [48] during the annotation process were proposed as a mechanism to foster deliberative…”
BackgroundAI image generators often give racist and sexist results: can they be fixed?
Ananya, A. (2024) · Nature · doi:10.1038/d41586-024-00674-9 · Cited by 50
Enhancing precision agriculture through cloud based transformative crop recommendation model
Singh, G., & Sharma, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-93417-3 · Cited by 76
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 102
Biological research and self-driving labs in deep space supported by artificial intelligence
Sanders, L., Scott, R. T., Yang, J. H., Qutub, A. A., Martín, H. G., Berrios, D. C., Hastings, J. J. A., Rask, J. C., Mackintosh, G., Hoarfrost, A., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00618-4 · Cited by 71
“To facilitate this, the community needs a set of standardized ontologies and data formatting guidelines specifically for space biology (for example, the inclusion of a datasheet to describe each dataset 105 ).”
MethodCertification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
Scharowski, N., Benk, M., Kühne, S. J., Wettstein, L., & Brühlmann, F. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593994 · Cited by 74
“Various methods have been proposed to increase AI systems’ transparency and, thereby auditability, such as through the use of model documentation or information about datasets [14, 21].”
MethodA.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities
Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., & Yang, J. (2024) · ACM Computing Surveys · doi:10.1145/3665926 · Cited by 98
“Other works build tools, e.g., documentation or checklists, [7, 29, 74, 144] and workflows [205] to support making and documenting relevant choices when building or evaluating a model.”
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Anuoluwapo, A., Bosselut, A., Chandu, K. R., Clinciu, M., Das, D., Dhole, K. D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.01672 · Cited by 49
“cards (Bender and Friedman, 2018; Gebru et al., 2018) we developed and for which we release a template and guide.”
BackgroundMethodInfluential
Metadata
- Authors
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K.
- Affiliations
- DAIR Institute (US)
- University of Washington (US)
- Cornell University (US)
- Microsoft Research New York City (United States) (US)
- University of Maryland, College Park, MD
- University of Southern California (US)
- ORCID
- Venue
- Communications of the ACM
- Published
- 2021-11-19
- Volume
- 64
- Issue
- 12
- Pages
- 86-92
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2795038878
- Semantic Scholar
- 0df347f5e3118fac7c351917e3a497899b071d1e
- Cited by
- 2,469
- References
- 37
- Citations by year
- 2026: 788 · 2025: 402 · 2024: 298 · 2023: 379 · 2022: 221 · 2021: 217
- FWCI
- 252.30
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Alfred P. Sloan Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Keywords
- dataset documentation, data provenance, data sharing, data reuse, metadata
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.