Datasheets for datasets
Open accessThe DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Abstract
Documentation to facilitate communication between dataset creators and consumers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Paradata needs may include the preservation of novel forms of information, such as datasheets explaining training datasets or model cards documenting an AI tool, and of these documents' relation to archival collections processed using them [ 26 ].”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [26]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Significant critiques of LLMs have been raised, including concerns about bias and fairness, misinformation, opaqueness due to a lack of interpretability and explainability, and environmental impact (Bender et al., 2021; Gebru et al., 2021; Strubell et al., 2019).”
From Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments (2023) · cites this paper as Gebru et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If ML driven algorithms are considered as epistemic resources that help us to improve understanding and/or make better predictions, then the statistical nature of ML de facto prevents groups underrepresented in the data from contributing to this epistemic resource on an equal ground to dominant groups (Gebru et al., 2021)”
From Algorithmic profiling as a source of hermeneutical injustice (2025) · cites this paper as Gebru et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The authors found that providing examples of what the requirements can look like helps to guide practitioners in clarifying their own rules.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“I show how archival theories have been animated in documenting data in the context of AI and ML, including through documentation frameworks like Datasheets for Datasets”
From Can AI provenance inform archival provenance? (2026) · cites this paper as Gebru et al. 2021
12 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,822
Minimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109
(2018) · Cited by 3,510
Self-citationA sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts
Pang, B., & Lee, L. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1218990 · Cited by 3,301
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Researchers at Google published follow-up work on model cards that document machine learning models [20] and released a data card (a lightweight version of a datasheet) along with the Open Images dataset [17].”
BackgroundSelf-citationMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 2×
“Of particular concern are recent examples showing that machine learning models can reproduce or amplify unwanted societal biases reflected in training data [4, 5, 9].”
BackgroundThe Effects of Automobile Safety Regulation
Peltzman, S. (1975) · Journal of Political Economy · doi:10.1086/260352 · Cited by 1,531
The Recent Past and Near Future of Risk and/or Need Assessment
Andrews, D. A., Bonta, J., & Wormith, J. S. (2006) · Crime & Delinquency · doi:10.1177/0011128705281756 · Cited by 1,507 · cited 1×
“Domains such as criminal justice (Garvie et al., 2016; Systems, 2017; Andrews et al., 2006), hiring and employment (Mann & O’Neil, 2016), critical infrastructure (O’Connor, 2017; Chui, 2017), and finance (Lin, 2012) all increasingly depend on machine learning methods.”
Background
Research works printed but not linked: 7
- Chui, G. Project will use AI to prevent or minimize electric grid failures, 2017.
- Garvie, C., Bedoya, A., and Frankle, J. The Perpetual Line-Up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, Washington, D.C., 2016.
- Holstein, K., Vaughan, J., Daumé, H, Dudík, M., and Wallach, H. Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of 2019 ACM CHI Conf. on Human Factors in Computing Systems.
- Krasin, I. et al. OpenImages: A public dataset for large-scale multi-label and multi-class image classification, 2017.
- O'Connor, M. How AI Could Smarten Up Our Water System, 2017.
- Doha Supply Systems. Facial Recognition, 2017.
- Yagcioglu, S., Erdem, A., Erdem, E., and Ikizler-Cinbis, N. RecipeQA: A challenge dataset for multimodal comprehension of cooking recipes. In Proceedings of the 2018 Conf. on Empirical Methods in Natural Language Processing.
Cited by
Items 301–400 of 1,039
Symphony: Composing Interactive Interfaces for Machine Learning
Bäuerle, A., Cabrera, Á. A., Hohman, F., Maher, M., Koski, D., Suau, X., Barik, T., & Moritz, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502102 · Cited by 69
Chen, J., Shao, Z., Zheng, X., Zhang, K., & Yin, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-53318-3 · Cited by 95
Analyzing Dataset Annotation Quality Management in the Wild
Klie, J.-C., de Castilho, R. E., & Gurevych, I. (2024) · Computational Linguistics · doi:10.1162/coli_a_00516 · Cited by 136
“These are, among others, datasheets for datasets (Gebru et al. 2021), dataset nutrition labels (Holland et al. 2018), data statements for NLP (Bender and Friedman 2018), accountability frameworks (Hutchinson et al. 2021), or data cards (Pushkarna, Zaldivar, and Kjartansson 2022).”
BackgroundCao, Y. T., & Daumé, H. (2021) · Computational Linguistics · doi:10.1162/coli_a_00413 · Cited by 47
“com/TristaCao/into_inclusivecoref with corresponding datasheets (Gebru et al. 2018).”
BackgroundSelf-citationHagendorff, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09573-8 · Cited by 47
“Current approaches to improve datasheets for datasets also do not include aspects that go beyond technical and organizational items like “Who funded the creation of the datasets?”, “Are there any errors, sources of noise, or redundancies in the dataset?”, “Will the datasets be updated?”, and so on (Gebru et al., 2018).”
BackgroundRamesh, D., Kameswaran, V., Wang, D., & Sambasivan, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533237 · Cited by 47
“Under a dynamic where users express skepticism, and seek to take action towards accountability, transparency (i.e., of models, datasets and practices surrounding the development of ADS) are viable mechanisms [53, 64, 89].”
BackgroundInfluentialWhose side are ethics codes on? power, responsibility and the social good
Washington, A. L., & Kuo, R. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372844 · Cited by 59
Blagec, K., Kraiger, J., Frühwirt, W., & Samwald, M. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104274 · Cited by 47
“[15] Several initiatives to index datasets and benchmark datasets relevant to AI research have been introduced in recent years.”
BackgroundWeb scraping for research: Legal, ethical, institutional, and scientific considerations
Brown, M. A., Gruen, A., Maldoff, G., Messing, S., Sanderson, Z., & Zimmer, M. T. (2025) · Big Data & Society · doi:10.1177/20539517251381686 · Cited by 108
“This may include limiting the scope of data shared, specifying licensing restrictions, maintaining clear documentation about the dataset ’ s intended use (Gebru et al., 2021), and avoiding public release when the risk of misuse is high.”
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning
Boratko, M., Li, X. L., O’Gorman, T., Das, R., Le, D., & McCallum, A. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.85 · Cited by 46
The unbearable (technical) unreliability of automated facial emotion recognition
Cabitza, F., Campagner, A., & Mattioli, M. (2022) · Big Data & Society · doi:10.1177/20539517221129549 · Cited by 69
“More in general, our results resonate with recent initiatives (Bender and Friedman, 2018; Gebru et al., 2021; Holland et al., 2018) aimed at raising awareness about the data production process (Gitelman, 2013), including the need to document in which (technical, social, economical and political)…”
ResultRacial Bias within Face Recognition: A Survey
Yücer, Ş., Tektas, F., Moubayed, N. A., & Breckon, T. P. (2024) · ACM Computing Surveys · doi:10.1145/3705295 · Cited by 81
“Furthermore, transparent provision of the ethical considerations together with any details of the racial annotation process in use and the intended possible use cases, limitations, and risks of the designed solution, should be made by the originating team in all cases [47].”
Artificial Intelligence in Industry 4.0: A Review of Integration Challenges for Industrial Systems
Windmann, A., Wittenberg, P., Schieseck, M., & Niggemann, O. (2024) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin58382.2024.10774364 · Cited by 82
McFee, B., Kim, J. W., Cartwright, M., Salamon, J., Bittner, R., & Bello, J. P. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2018.2875349 · Cited by 68
“proposed a standardized “datasheet” format for general machine learning datasets, inspired by the standardized datasheets that accompany electronic components [48].”
BackgroundTutorial: Safe and Reliable Machine LearningPreprint
Saria, S., & Subbaswamy, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07204 · Cited by 107
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-TuningPreprint
Chae, Y., & Davidson, T. (2023) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/sthwk · Cited by 71
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Data Governance in the Age of Large-Scale Data-Driven Language Technology
Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534637 · Cited by 75
“In practice, compliance is hard to enforce, and while datasets are increasingly accompanied by datasheets [59] and similar documentation, navigating the legal structures involved is not always straightforward.”
BackgroundMittal, S., Thakral, K., Singh, R., Vatsa, M., Glaser, T., Ferrer, C. C., & Hassner, T. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00874-y · Cited by 79
“The discourse surrounding dataset quality in AI research is evolving, encompassing both qualitative and quantitative evaluation methods 5,6 .”
InfluentialRaghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603207 · Cited by 49
Distrust of Artificial Intelligence: Sources & Responses from Computer Science & Law
Dwork, C., & Minow, M. (2022) · Daedalus · doi:10.1162/daed_a_01918 · Cited by 44
“25 But disclosure of learning algorithms themselves has limited usefulness in the absence of data features with comprehensible meanings and explanations of weight determining the contribution of each feature to outcomes.”
BackgroundEhsan, U., Singh, R., Metcalf, J., & Riedl, M. O. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533186 · Cited by 73
“Exemplars of the technical perspective include developing technical documentation and organizational accountability practices [39, 73, 86], audits of the outputs of algorithmic systems [17, 80], demonstrations of the perils of large datasets [11, 48, 56], and formalized models of algorithmic fairness [10, 77].”
BackgroundMelhárt, D., Togelius, J., Mikkelsen, B., Holmgård, C., & Yannakakis, G. N. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3276425 · Cited by 79
“On the other hand, systems relying on historical biases can propagate patterns that seem true to a casual observer and are only revealed as biased under a more critical analysis [99], [100].”
BackgroundSourlos, N., Vliegenthart, R., Santinha, J., Klontzas, M. E., Cuocolo, R., Huisman, M., & van Ooijen, P. M. A. (2024) · Insights into Imaging · doi:10.1186/s13244-024-01833-2 · Cited by 56
“Relevant clinical and demographic information should also be made available to allow subgroup analysis [60].”
BackgroundExperiences with Improving the Transparency of AI Models and Services
Hind, M., Houde, S., Martino, J., Mojsilovic, A., Piorkowski, D., Richards, J., & Varshney, K. R. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3383051 · Cited by 44
“Recent work has outlined the need for increased transparency in AI for data sets (Gebru et al. 2018; Bender and Friedman 2018; Holland et al. 2018), models (Mitchell et al. 2019), and services (Arnold et al. 2019).”
BackgroundAn Examination of Fairness of AI Models for Deepfake Detection
Trinh, L., & Liu, Y. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/79 · Cited by 44
The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design
Shen, H., Wang, L., Deng, W. H., Brusse, C., Velgersdijk, R., & Zhu, H. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533110 · Cited by 64
Barriers to computer vision applications in pig production facilities
Li, J., Green-Miller, A. R., Hu, X., Lucic, A., Mohan, M. R. M., Dilger, R. N., Condotta, I. C. F. D. S., Aldridge, B. M., Hart, J. M., & Ahuja, N. (2022) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2022.107227 · Cited by 57
Data Hunches: Incorporating Personal Knowledge into Visualizations
Lin, H., Akbaba, D., Meyer, M., & Lex, A. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209451 · Cited by 57
“In scientific data analysis, for example, caveats about the data are often captured in (digital or paper) lab notebooks, as comments in analysis scripts, and as datasheets recorded alongside a dataset [26], and they are discussed with colleagues in lab meetings, and reported to a general audience in methods sections o…”
BackgroundMethodologies for Monitoring Mental Health on Twitter: Systematic Review
Cara, N. H. D., Maggio, V., Davis, O. S. P., & Haworth, C. M. A. (2023) · Journal of Medical Internet Research · doi:10.2196/42734 · Cited by 57
“We particularly recommend the use of Data Sheets for Datasets [129] for thorough data set reporting, which can be included as supplementary material hosted on a web-based repository that provides a permanent digital object identifier (DOI), such as the Open Science Framework or a preprint server.”
Malgieri, G., & Pasquale, F. (2023) · Computer law & security review · doi:10.1016/j.clsr.2023.105899 · Cited by 87
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Sun, J. J., Karigo, T., Chakraborty, D., Mohanty, S. P., Wild, B., Sun, Q., Chen, C., Anderson, D. J., Perona, P., Yue, Y., et al. (2021) · PubMed · doi:10.48550/arxiv.2104.02710 · Cited by 82
“• Section B contains dataset documentation and intended uses for CalMS21, following the format of the Datasheet for Datasets[20].”
BackgroundMethodInfluentialWolfe, R., Yang, Y., Howe, B., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594072 · Cited by 42
“Mitigation Strategies Documenting, curating, and preprocessing training datasets (Gebru et al., 2021) to remove problematic content may help to prevent the propagation of bias to zero-shot and downstream settings.”
BackgroundKubsch, M., Krist, C., & Rosenberg, J. M. (2022) · Journal of Research in Science Teaching · doi:10.1002/tea.21803 · Cited by 53
Integrating ESG and AI: a comprehensive responsible AI assessment framework
Lee, S. U., Perera, H., Liu, Y., Xia, B., Lu, Q., Zhu, L., Cairns, J., & Nottage, M. (2025) · AI and Ethics · doi:10.1007/s43681-025-00741-5 · Cited by 95
Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems
Lewis, G. A., Bellomo, S., & Özkaya, İ. (2021) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/wain52551.2021.00028 · Cited by 58
What Did My AI Learn? How Data Scientists Make Sense of Model Behavior
Cabrera, Á. A., Ribeiro, M. T., Lee, B., DeLine, R. A., Perer, A., & Drucker, S. M. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3542921 · Cited by 62
“For example, Datasheets for Datasets [56] tracks the metadata of a dataset, such as provenance and demographic distribution, to inform future model builders, while Model Cards [27] describe AI models to inform their use and potential downsides.”
BackgroundWhere is the Human?: Bridging the Gap Between AI and HCI
Inkpen, K., Chancellor, S., De Choudhury, M., Veale, M., & Baumer, E. P. S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3299002 · Cited by 54
Large language models and political science
Linegar, M., Kocielnik, R., & Alvarez, R. M. (2023) · Frontiers in Political Science · doi:10.3389/fpos.2023.1257092 · Cited by 91
Seamful XAI: Operationalizing Seamful Design in Explainable AI
Ehsan, U., Liao, Q. V., Passi, S., Riedl, M. O., & Daumé, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637396 · Cited by 105
“Research also aims to provide structured yet flexible “scaffolding”, such as guiding frameworks [21, 28, 59], checklists [57], processes [50, 70] and tools to lower the barriers and empower individuals to engage in RAI practices [35, 50, 57].”
BackgroundStart Generating: Harnessing Generative Artificial Intelligence for Sociological Research
Davidson, T. (2024) · Socius Sociological Research for a Dynamic World · doi:10.1177/23780231241259651 · Cited by 69
“Computer scientists have proposed including documentation with models and training datasets (Gebru et al. 2021; Mitchell et al. 2019) that indicates any known or latent biases.”
Bischl, B., Casalicchio, G., Feurer, M., Gijsbers, P., Hutter, F., Lang, M., Mantovani, R. G., van Rijn, J. V., & Vanschoren, J. (2017) · NeurIPS Datasets and Benchmarks · doi:10.48550/arxiv.1708.03731 · Cited by 280
“In the future we want to support this natively on a dataset level (e.g., by integrating datasheets [Gebru et al., 2018]) and benchmark suite level (by providing a dedicated information field).”
BackgroundMachine Learning Explainability for External Stakeholders
Bhatt, U., Andrus, M., Weller, A., & Xiang, A. (2020) · arXiv · doi:10.48550/arxiv.2007.05408 · Cited by 41
“Though, making that information available could be nontrivial (Raji & Yang, 2019; Arnold et al., 2019; Gebru et al., 2018; Mitchell et al., 2019).”
BackgroundAbout Face: A Survey of Facial Recognition EvaluationPreprint
Raji, I. D., & Fried, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.00813 · Cited by 95
“Period Period I Period II Period III Period IV Years Before 1996 1996 - 2007 2007-2014 After 2014 Number of Datasets Created 5 37 33 45 Range of number of Images in a dataset (MIN- MAX) 56 - 14,126 120 - 121,589 154 - 750,000 642 - 50,000,000 Range of number of Subjects in a dataset (MIN- MAX) 4 - 1,199 10 - 37,437 32…”
BackgroundInfluentialAI Regulation Is (not) All You Need
Lucaj, L., van der Smagt, P., & Benbouzid, D. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594079 · Cited by 40
“Datasheets are inspired by the standardised forms of information sharing and rigorous testing, established in the electronic hardware industry, to enable the overview of the components performances under different test conditions [30].”
InfluentialTan, M., & Subramonyam, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642592 · Cited by 40
“Researchers have proposed factsheets [8], datasheets [14, 36, 45], and model cards [76] to specify the content of AI documentation, including purpose, performance metrics, training data composition, and disclosures of limitations.”
BackgroundGetting AI Right: Introductory Notes on AI & Society
Manyika, J. (2022) · Daedalus · doi:10.1162/daed_e_01897 · Cited by 79
Do, K., Pang, R. Y., Jiang, J., & Reinecke, K. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581347 · Cited by 62
“” Compared to existing specific AI-related checklist [28, 45, 79], a guideline can inform “at what point you know if we could potentially have some”
BackgroundBerman, G., Goyal, N., & Madaio, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642398 · Cited by 72
“In this context, we conceive of a dataset transparency protocol as an RAI tool designed to enable data scientists to document relevant aspects of a dataset in a standardized manner [e.g., 22, 77, 162].”
MethodAdler, D. A., Stamatis, C. A., Meyerhoff, J., Mohr, D. C., Wang, F., Aranovich, G. J., Sen, S., & Choudhury, T. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00057-y · Cited by 52
“More thorough reporting of model training data 51 , and mon-itoringAItoolsin “ silentmode ” ,inwhichpredictionsaremadebutnotused for decision making 52 , could prevent AI tools developed in dissimilar populations from causing harm.”
MethodBalakrishnan, M., Ferreira, K., & Tong, J. D. (2025) · Management Science · doi:10.1287/mnsc.2022.03850 · Cited by 87
How far are we from reproducible research on code smell detection? A systematic literature review
Lewowski, T., & Madeyski, L. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106783 · Cited by 48
Martí‐Bonmatí, L., Miguel, A., Suárez‐García, A., Aznar, M., Bérégi, J.-P., Fournier, L. S., Neri, E., Laghi, A., França, M., Sardanelli, F., et al. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.742701 · Cited by 50
“In this manner, this project further highlights the current need of datasheets for datasets (29) Regarding the Common Data Model (CDM) to be used for clinical data across clinical centers, the use of the OMOP (Observational Medical Outcomes Partnership) CDM (30) was agreed upon, which is based on standard ontologies s…”
MethodThree lines of defense against risks from AI
Schuett, J. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01811-0 · Cited by 41
“Reports will typically include risk registers and risk matrices (IEC 2019), but they could also involve information about specific models, in the form of (preliminary) model cards (Mitchell et al. 2019), data sheets (Gebru et al. 2021), and system cards (Green et al. 2022).”
AI-READI: rethinking AI data collection, preparation and sharing in diabetes research and beyond
Baxter, S. L., de Sa, V. R., Ferryman, K. S., Jain, P., Lee, C. S., Li‐Pook‐Than, J., Liu, T. Y. A., Owen, J. P., Patel, B. J., Yu, Q., et al. (2024) · Nature Metabolism · doi:10.1038/s42255-024-01165-x · Cited by 68
Reflections on Gender Analyses of Bibliographic Corpora
Mihaljević, H., Tullney, M., Santamaría, L., & Steinfeldt, C. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00029 · Cited by 43
“Recent work by Gebru et al. (2018) formalizes this in a sense by proposing a framework to document data sets with data sheets containing a list of standardized questions: why a data set was created, who funded it, what preprocessing has been done, and in case it relates to real people, whether they…”
BackgroundSoftcite dataset: A dataset of software mentions in biomedical and economic research publications
Du, C., Cohoon, J., Lopez, P., & Howison, J. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24454 · Cited by 56
“Further details of annotation, training, alignment, and curation are documented later in this paper in detail sufficient to understand the provenance of the dataset (Gebru et al., 2018; Thomer, Wickett, Baker, Fouke, & Palmer, 2018).”
BackgroundMethodResponsible AI: A Primer for the Legal Community
Golbin, I., Rao, A. S., Hadjarian, A., & Krittman, D. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9377738 · Cited by 53
Metaphors for designers working with AI
Murray-Rust, D., Nicenboim, I. P., & Lockton, D. (2022) · Proceedings of DRS · doi:10.21606/drs.2022.667 · Cited by 59
Data Representativeness in Accessibility Datasets: A Meta-Analysis
Kamikubo, R., Wang, L., Marte, C., Mahmood, A., & Kacorri, H. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3544826 · Cited by 64
Moore, S., Liao, Q. V., & Subramonyam, H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581242 · Cited by 51
“AI failures can have negative consequences and create physical, psychological, or financial harm to humans [27, 52].”
BackgroundA Primer for Evaluating Large Language Models in Social-Science Research
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251325174 · Cited by 98
“Researchers developing or fine-tuning LLMs should consider fully documenting their models using established frameworks such as model cards (Mitchell et al., 2019) and data sheets for data sets used in training and fine-tuning (Gebru et al., 2021).”
Collaborative Practices with Structured Data: Do Tools Support What Users Need?
Koesten, L. M., Kacprzak, E., Tennison, J., & Simperl, E. P. B. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300330 · Cited by 52
“Recently ideas for supplementary information (datasheets) that should be shared together with datasets have been suggested by [25].”
BackgroundSrinivasan, R., & Chander, A. (2021) · Queue · doi:10.1145/3466132.3466134 · Cited by 49
Sloane, M., & Zakrzewski, J. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533156 · Cited by 59
Ramaswamy, V., Kim, S. S. Y., Fong, R., & Russakovsky, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01052 · Cited by 99
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue ModelingPreprint
Lin, Z., Madotto, A., Winata, G. I., Xu, P., Jiang, F., Hu, Y., Chen, S., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02787 · Cited by 59
“[51] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna 481 Wallach, Hal Daumé III, and Kate Crawford.”
BackgroundMethodUsing ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments
Jungherr, A. (2023) · doi:10.31235/osf.io/d84q6 · Cited by 37
Natural language processing for clinical notes in dentistry: A systematic review
Pethani, F., & Dunn, A. G. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104282 · Cited by 54
Affective Game Computing: A Survey
Yannakakis, G. N., & Melhárt, D. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3315689 · Cited by 60
“Biased affect models can encode and perpetuate socioeconomic and sociopolitical biases that lead to direct or indirect harm to the users [233]– [235].”
BackgroundFACET: Fairness in Computer Vision Evaluation Benchmark
Gustafson, L., Rolland, C., Ravi, N., Duval, Q., Adcock, A., Fu, C.-Y., Hall, M. A., & Ross, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01863 · Cited by 95
“Lastly, FACET and other fairness datasets are representative of the current time period and organizational infrastructure within which it was created [18, 32].”
BackgroundRidley, M. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24889 · Cited by 74
“Vaughan and Wallach (2021) argue for including explanations of datasets used as training data and system performance metrics, typically used only in algorithmic XAI, as key aspects of HCXAI.”
Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data
Zhang, C., & Wang, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00780 · Cited by 91
Cajueiro, D. O., & Celestino, V. R. R. (2025) · Journal of Economy and Technology · doi:10.1016/j.ject.2025.07.001 · Cited by 67
Towards trustworthy AI in Industry 5.0: Ante-hoc interpretability with deep learning
Zhou, L., Chen, L., Zhu, T., Su, X., Dong, Y., Wen, L., Liu, H., Lu, X., Leng, J., Liu, Q., et al. (2026) · Computers & Industrial Engineering · doi:10.1016/j.cie.2026.111805 · Cited by 55
Clinician's guide to trustworthy and responsible artificial intelligence in cardiovascular imaging
Szabó, L. E., Raisi‐Estabragh, Z., Salih, A. M., McCracken, C., Pujadas, E. R., Gkontra, P., Kiss, M., Maurovich‐Horvat, P., Vágó, H., Merkely, B. P., et al. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1016032 · Cited by 45
“posed a list of questions on how and why data was collected, what is the composition of the data, and how it was curated and labeled in their document entitled "Datasheets for Datasets” (102).”
Backgroundvan Berkel, N., Bellio, M., Skov, M. B., & BLANDFORD, A. E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3571815 · Cited by 47
“[36], which proposes that each dataset is accompanied by documentation, including inter alia its motivation, data collection process, and recommended use.”
BackgroundThe Problem with Metrics is a Fundamental Problem for AIPreprint
Thomas, R., & Uminsky, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08512 · Cited by 37
(2021) · doi:10.18653/v1/2021.naacl-main.46 · Cited by 115
Chen, C., & Sundar, S. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580805 · Cited by 49
Epistemically violent biases in artificial intelligence design: the case of DALLE-E 2 and Starry AI
Mbalaka, B. (2023) · Digital Transformation and Society · doi:10.1108/dts-01-2023-0003 · Cited by 49
The Open Kidney Ultrasound Data Set
Singla, R., Ringstrom, C., Hu, G., Lessoway, V. A., Reid, J., Nguan, C. Y. C., & Rohling, R. N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44521-7_15 · Cited by 68
AI Consent Futures: A Case Study on Voice Data Collection with Clinicians
Wilcox, L. G., Brewer, R. N., & Díaz, F. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610107 · Cited by 45
“This finding suggests the importance of future research concerned with data and system transparency in multiple forms: from the composition of a data set [67, 131] to methods for interactive, user-controllable, and multi-modal forms of system output.”
BackgroundNg, M. Y., Youssef, A., Miner, A. S., Sarellano, D., Long, J., Larson, D. B., Hernandez‐Boussard, T., & Langlotz, C. P. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.45892 · Cited by 47
Nutritional Labels for Data and Models
Stoyanovich, J., & Howe, B. (2019) · IEEE Data(base) Engineering Bulletin · Cited by 37
AI and Holistic Review: Informing Human Reading in College Admissions
Alvero, A. J., Arthurs, N., antonio, A. L., Domingue, B. W., Gebre-Medhin, B., Giebel, S., & Stevens, M. L. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375871 · Cited by 35
Leins, K., Lau, J. H., & Baldwin, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.261 · Cited by 45
Human Movement Datasets: An Interdisciplinary Scoping Review
Olugbade, T., Bieńkiewicz, M. M. N., Barbareschi, G., D’Amato, V., Oneto, L., Camurri, A., Holloway, C., Björkman, M., Keller, P. E., Clayton, M. R. L., et al. (2022) · ACM Computing Surveys · doi:10.1145/3534970 · Cited by 48
“This is important to ensure that secondary uses respect the limitations of the datasets, although targeting dataset creators for machine learning in the industry, [35] contains guidelines and examples that can be useful to the wider research community for creating such documentations.”
BackgroundMethodInfluentialNorval, C., Cornelius, K. B., Cobbe, J., & Singh, J. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533133 · Cited by 71
“datasheets [21], model cards [27]), or any number”
BackgroundDocumenting High-Risk AI: A European Regulatory Perspective
Hupont, I., Micheli, M., Delipetrev, B., Gómez, E., & Garrido, J. S. (2023) · Computer · doi:10.1109/mc.2023.3235712 · Cited by 46
Lara, M. A. R., Kowalczuk, M. V. R., Eliceche, M. L., Ferraresso, M. G., Luna, D. R., Benítez, S., & Mazzuoccolo, L. D. (2023) · Scientific Data · doi:10.1038/s41597-023-02630-0 · Cited by 48
“…which aims to provide detailed information on the processes of design, compilation, and refinement of this database, as well as the description of its final content, to ensure the necessary transparency demanded by the community through different initiatives, guidelines, and frameworks 20–23 .”
BackgroundRaji, I. D., & Yang, J. (2019) · arXiv · doi:10.48550/arxiv.1912.06166 · Cited by 36
“…industry [4] or nutrition labels from the food industry [7], the suggested templates vary widely in length and appearance, ranging from a single concise page of succinct statements [11] or a set of symbols and visualizations [7, 10] to upwards of 10 pages of detailed prose and graphs [4].”
BackgroundInfluentialThe Emerging Generative Artificial Intelligence Divide in the United States
Daepp, M. I. G., & Counts, S. (2025) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v19i1.35825 · Cited by 75
MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data Science
Sun, C., Asudeh, A., Jagadish, H. V., Howe, B., & Stoyanovich, J. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3357853 · Cited by 44
“Other follow-on works propose similar, manually constructed, data sheets [15] or model cards [16].”
BackgroundTeaching Responsible Data Science: Charting New Pedagogical Territory
Lewis, A., & Stoyanovich, J. (2021) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00241-7 · Cited by 62
Hullman, J. R., Kapoor, S., Nanayakkara, P., Gelman, A., & Narayanan, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534196 · Cited by 59
Hupont, I., Tolan, S., Güneş, H., & Gómez, E. (2022) · Scientific Reports · doi:10.1038/s41598-022-14981-6 · Cited by 56
Lee, V. R., Pimentel, D. R., Bhargava, R., & D’Ignazio, C. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13251 · Cited by 39
A Transparency Index Framework for AI in EducationPreprint
Chaudhry, M. A., Cukurova, M., & Luckin, R. (2022) · EdArXiv (OSF Preprints) · doi:10.35542/osf.io/bstcf · Cited by 82
Watkins, E. A., McKenna, M., & Chen, J. (2022) · arXiv · doi:10.48550/arxiv.2202.09519 · Cited by 35
Metadata
- Authors
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K.
- Affiliations
- DAIR Institute (US)
- University of Washington (US)
- Cornell University (US)
- Microsoft Research New York City (United States) (US)
- University of Maryland, College Park, MD
- University of Southern California (US)
- ORCID
- Venue
- Communications of the ACM
- Published
- 2021-11-19
- Volume
- 64
- Issue
- 12
- Pages
- 86-92
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2795038878
- Semantic Scholar
- 0df347f5e3118fac7c351917e3a497899b071d1e
- Cited by
- 2,469
- References
- 37
- Citations by year
- 2026: 788 · 2025: 402 · 2024: 298 · 2023: 379 · 2022: 221 · 2021: 217
- FWCI
- 252.30
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Alfred P. Sloan Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Keywords
- dataset documentation, data provenance, data sharing, data reuse, metadata
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.