Datasheets for datasets
Open accessThe DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Abstract
Documentation to facilitate communication between dataset creators and consumers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Paradata needs may include the preservation of novel forms of information, such as datasheets explaining training datasets or model cards documenting an AI tool, and of these documents' relation to archival collections processed using them [ 26 ].”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [26]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Significant critiques of LLMs have been raised, including concerns about bias and fairness, misinformation, opaqueness due to a lack of interpretability and explainability, and environmental impact (Bender et al., 2021; Gebru et al., 2021; Strubell et al., 2019).”
From Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments (2023) · cites this paper as Gebru et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If ML driven algorithms are considered as epistemic resources that help us to improve understanding and/or make better predictions, then the statistical nature of ML de facto prevents groups underrepresented in the data from contributing to this epistemic resource on an equal ground to dominant groups (Gebru et al., 2021)”
From Algorithmic profiling as a source of hermeneutical injustice (2025) · cites this paper as Gebru et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The authors found that providing examples of what the requirements can look like helps to guide practitioners in clarifying their own rules.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“I show how archival theories have been animated in documenting data in the context of AI and ML, including through documentation frameworks like Datasheets for Datasets”
From Can AI provenance inform archival provenance? (2026) · cites this paper as Gebru et al. 2021
12 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,822
Minimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109
(2018) · Cited by 3,510
Self-citationA sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts
Pang, B., & Lee, L. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1218990 · Cited by 3,301
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Researchers at Google published follow-up work on model cards that document machine learning models [20] and released a data card (a lightweight version of a datasheet) along with the Open Images dataset [17].”
BackgroundSelf-citationMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 2×
“Of particular concern are recent examples showing that machine learning models can reproduce or amplify unwanted societal biases reflected in training data [4, 5, 9].”
BackgroundThe Effects of Automobile Safety Regulation
Peltzman, S. (1975) · Journal of Political Economy · doi:10.1086/260352 · Cited by 1,531
The Recent Past and Near Future of Risk and/or Need Assessment
Andrews, D. A., Bonta, J., & Wormith, J. S. (2006) · Crime & Delinquency · doi:10.1177/0011128705281756 · Cited by 1,507 · cited 1×
“Domains such as criminal justice (Garvie et al., 2016; Systems, 2017; Andrews et al., 2006), hiring and employment (Mann & O’Neil, 2016), critical infrastructure (O’Connor, 2017; Chui, 2017), and finance (Lin, 2012) all increasingly depend on machine learning methods.”
Background
Research works printed but not linked: 7
- Chui, G. Project will use AI to prevent or minimize electric grid failures, 2017.
- Garvie, C., Bedoya, A., and Frankle, J. The Perpetual Line-Up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, Washington, D.C., 2016.
- Holstein, K., Vaughan, J., Daumé, H, Dudík, M., and Wallach, H. Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of 2019 ACM CHI Conf. on Human Factors in Computing Systems.
- Krasin, I. et al. OpenImages: A public dataset for large-scale multi-label and multi-class image classification, 2017.
- O'Connor, M. How AI Could Smarten Up Our Water System, 2017.
- Doha Supply Systems. Facial Recognition, 2017.
- Yagcioglu, S., Erdem, A., Erdem, E., and Ikizler-Cinbis, N. RecipeQA: A challenge dataset for multimodal comprehension of cooking recipes. In Proceedings of the 2018 Conf. on Empirical Methods in Natural Language Processing.
Cited by
Items 701–800 of 1,039
Geng, Y. (2024) · Entertainment Computing · doi:10.1016/j.entcom.2024.100816 · Cited by 14
Aboelazm, K. S. (2025) · International Journal of Law and Management · doi:10.1108/ijlma-06-2024-0208 · Cited by 23
Easy, Reproducible and Quality-Controlled Data Collection with CROWDAQ
Ning, Q., Wu, H., Dasigi, P., Dua, D., Gardner, M., Logan, R. L., Marasović, A., & Nie, Z. (2020) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2020.emnlp-demos.17 · Cited by 12
HumBugDB: A Large-scale Acoustic Mosquito Dataset
Kiskin, I., Sinka, M. E., Cobb, A. D., Rafique, W., Wang, L., Zilli, D., Gutteridge, B., Dam, R., Marinos, T., Li, Y., et al. (2021) · Warwick Research Archive Portal (University of Warwick) · doi:10.48550/arxiv.2110.07607 · Cited by 30
“We follow the structure outlined in Datasheets for Datasets by Gebru et al. [2018].”
MethodThe ROOTS Search Tool: Data Transparency for LLMs
Piktus, A., Akiki, C., Villegas, P., Laurençon, H., Dupont, G., Luccioni, A. S., Jernite, Y., & Rogers, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.29 · Cited by 59
Artificial Intelligence Strategies for National Security and Safety StandardsPreprint
Blasch, E., Sung, J., Nguyen, T., Daniel, C. P., & Mason, A. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05727 · Cited by 25
“These efforts include: Datasheets [42], data statements [43], and “nutrition la-bels” [44] to better understand the data that underlies a machine learning system; Cards to clarify the intended use cases of machine learning algorithms and minimize their usage in contexts for which they are not…”
BackgroundMeasuring justice in machine learning
Lundgard, A. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372838 · Cited by 25
“That is, not only does a measure achieve sensitivity to people’s heterogeneities with respect to particular extant datasets (Gebru et al., 2020), it aims to achieve sensitivity across the entire data pipeline: who collects the data, what data is collected, how data is being collected, and which calculations and evalua…”
Rissanen Data Analysis: Examining Dataset Characteristics via Description LengthPreprint
Perez, E., Kiela, D., & Cho, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03872 · Cited by 16
“Our work opens up ample opportunity for future work: automatically uncovering dataset biases when writing data statements (Gebru et al., 2018; Bender and Friedman, 2018), selecting the datasets to include in future benchmarks, discovering which capabilities are helpful for different tasks, and also…”
BackgroundResultESR: Ethics and Society Review of Artificial Intelligence ResearchPreprint
Bernstein, M. S., Levi, M., Magnus, D. C., Rajala, B., Satz, D. M., & Waeiss, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11521 · Cited by 30
“Progress is being made in many dimensions, including algorithmic advances [2, 8, 26, 36], norm changes [14], improved product design processes [30, 32, 38, 50, 56, 64, 77, 81], and both academic and public activism on these issues [9].”
BackgroundInfluentialMundt, M., Lang, S., Delfosse, Q., & Kersting, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03331 · Cited by 29
“…Hutter, 2020), human-centered machine learning model cards (Mitchell et al., 2019), or general dataset sheet specifications (Bender & Friedman, 2018; Gebru et al., 2018), which all make valuable propositions to overcome existing gaps through the creation of standardized best-practice (check-)lists.”
BackgroundResultInfluentialGrill, G. (2022) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/zekqv · Cited by 19
Artificial Intelligence for Earth System Predictability (AI4ESP) (2021 Workshop Report)
Hickmon, N. L., Varadharajan, C., Hoffman, F. M., Wainwright, H. M., & Collis, S. (2022) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/1888810 · Cited by 19
Tan, M., Lee, H., Wang, D., & Subramonyam, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637358 · Cited by 14
“Research in data-centric AI has primarily addressed the downstream harms of low-quality data through the creation of numerous frameworks for facilitating data accountability and transparency through clear documentation practices [3, 7, 15, 18, 25, 68, 71, 93].”
BackgroundInfluentialEcosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14
Knowledge Workers' Perspectives on AI Training for Responsible AI Use
Zhang, A., & Lee, M. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714100 · Cited by 25
“Different tools have also been created to facilitate critical reflection about responsible AI amongst the practitioners developing or engineering these systems [8, 25, 45].”
Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipeline
Kapania, S., Ballard, S. A., Kessler, A., & Vaughan, J. W. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732005 · Cited by 29
“While there are plenty of resources for data and model documentation [9, 14, 25, 66], there is almost no existing guidance on what to document for synthetic data that is created and used in such myriad ways.”
Self-citationSurvey on the Evaluation of Generative Models in Music
Lerch, A., Arthur, C., Bryan–Kinns, N., Ford, C., Sun, Q., & Vinay, A. (2025) · ACM Computing Surveys · doi:10.1145/3769106 · Cited by 39
“…for the holistic evaluation of a machine learning system [132], as the call for “dataset audit cards” [21] and formalized datasheets for datasets [93, 212] emphasize; (iv) resource use [19, 51, 215]: the high energy consumption of today’s models can be linked to non-negligible environmental…”
An HCAI Methodological Framework: Putting it Into Action to Enable Human-Centered AI
Xu, W., Gao, Z., & Dainoff, M. J. (2026) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3631590 · Cited by 96
“Inspired by joint cognitive systems theory [104], multiagent systems theory [127], and ecosystem theories [126], [128], Xu et al., [120] proposed a conceptual framework of human-AI joint cognitive ecosystems.”
Deep learning in acute vertigo diagnosis
Rastall, D. P. W., & Green, K. E. (2022) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2022.120454 · Cited by 13
Analysis: Flawed Datasets of Monkeypox Skin Images
Vega, C., Schneider, R., & Satagopam, V. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-01928-1 · Cited by 16
“To sum up, better documentation of the data acquisition process, causal assumptions, data annotation and curation protocols, help to mitigate and identify potential sources of bias and prevent data cascades [3, 58].”
BackgroundPradhan, S. K., Heyn, H.-M., & Knauss, E. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09622-8 · Cited by 19
“Other data management frameworks, such as datasheets for datasets proposed by Gebru et al. (2021), do not explicitly connect data quality attributes to data requirements.”
What’s the Meaning of Superhuman Performance in Today’s NLU?
Tedeschi, S., Bos, J., Declerck, T., Hajič, J., Hershcovich, D., Hovy, E. H., Koller, A., Krek, S., Schockaert, S., Sennrich, R., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.697 · Cited by 57
Alderman, J. E., Charalambides, M., Sachdeva, G. S., Laws, E., Palmer, J., Lee, E., Menon, V., Malik, Q., Vadera, S., Calvert, M., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00146-8 · Cited by 17
Auditing Flood Vulnerability Geo-Intelligence Workflow for Biases
Masinde, B. K., Gevaert, C. M., Nagenborg, M., den Homberg, M. J. C. V., Margutti, J., Gortzak, I., & Zevenbergen, J. A. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13120419 · Cited by 21
“Drawing inspiration from the auditing framework Scoping, Mapping, Artifact Collection, Testing and Reflection (SMACTR), we use model cards proposed by Mitchell et al. [19] for transparent AI model reporting and datasheets by Gebru et al. [20] for detailed data documentation.”
AI Development for the Public Interest: From Abstraction Traps to Sociotechnical Risks
Andrus, M., Dean, S., Gilbert, T. K., Lambert, N., & Zick, T. (2020) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas50296.2020.9462193 · Cited by 18
It’s the Meaning That Counts: The State of the Art in NLP and Semantics
Hershcovich, D., & Donatelli, L. E. (2021) · Künstliche Intell. · doi:10.1007/s13218-021-00726-6 · Cited by 17
“Initiatives such as data statements [13] and datasheets for datasets [63] facilitate transparency and accountability by making it clear to dataset consumers what the characteristics of the data are.”
BackgroundDesigning a Risk Assessment Tool for Artificial Intelligence Systems
Nagbøl, P. R., Müller, O., & Krancher, O. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-82405-1_32 · Cited by 22
“While most AI research does not explicitly draw on risk management theory [13, 14], the risk management literature does not focus on AI, examining instead risks associated with information system (IS) projects [8, 9] or with traditional software and hardware [22].”
BackgroundMethodInfluentialArtificial Intelligence Project Success Factors: Moral Decision-Making with Algorithms
Miller, G. J. (2021) · Annals of Computer Science and Information Systems · doi:10.15439/2021f26 · Cited by 22
MS-Mentions: Consistently Annotating Entity Mentions in Materials Science Procedural Text
O’Gorman, T., Jensen, Z., Mysore, S., Huang, K., Mahbub, R., Olivetti, E., & McCallum, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.101 · Cited by 14
“Following recommended practice to report finer details of the a dataset, we provide a Datasheet following prompts provided in Bender and Friedman (2018) and Gebru et al. (2018).”
MethodChandrasekhar, R., Utts, J. M., Cohen, A. I., & Hayat, M. J. (2022) · The American Statistician · doi:10.1080/00031305.2022.2156612 · Cited by 21
“There are many recent examples of work regarding ethical issues in this big data era (Dwork 2011; Barocas and Nissenbaum 2014; De Veaux et al. 2017; O’Neil 2017; Hand 2018; Tractenberg 2019; Mittelstadt and Kwakkel 2020; Nissenbaum 2020; Washington and Kuo 2020; Gebru et al. 2021).”
BackgroundBiases, Fairness, and Implications of Using AI in Social Media Data Mining
Anzum, F., Asha, A. Z., & Gavrilova, M. L. (2022) · International Conference on Cyberworlds (CW) · doi:10.1109/cw55638.2022.00056 · Cited by 22
“As proposed in [17] and [7], OSM researchers could adopt the practice of introducing datasheets for datasets.”
BackgroundTowards Traceability in Data Ecosystems using a Bill of Materials ModelPreprint
Barclay, I., Preece, A. D., Taylor, I., & Verma, D. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04253 · Cited by 38
“At the component level, Gebru et al explore the benefits of developing and maintaining Datasheets for Data[13], which replicates the specification documents that often accompany physical components, and Mitchell et al propose a document format for AI model specifications and benchmarks[14].”
BackgroundNon-portability of Algorithmic Fairness in IndiaPreprint
Sambasivan, N., Arnesen, E., Hutchinson, B., & Prabhakaran, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03659 · Cited by 32
“[22] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford.”
BackgroundFairness and Discrimination in Information Access SystemsPreprint
Ekstrand, M. D., Das, A., Burke, R., & Díaz, F. (2021) · arXiv (Cornell University) · Cited by 13
SoK: Machine Learning GovernancePreprint
Chandrasekaran, V., Jia, H., Thudi, A., Travers, A., Yaghini, M., & Papernot, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.10870 · Cited by 36
“[248] proposed datasheets for datasets, a generic documentation scheme meant to be used”
MethodBurr, C., & Leslie, D. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3937983 · Cited by 13
The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical ReviewPreprint
Morley, J. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/n7y2g · Cited by 13
Speaking of accent: A content analysis of accent misconceptions in ASR research
Prinos, K., Patwari, N., & Power, C. A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658969 · Cited by 12
“This reflection and metadata benefits transparency for the reader, but also benefits the researcher, in a similar vein to that described by Gebru et al. in [21].”
BackgroundMethodTime to audit your AI algorithms
Sandu, I., Wiersma, M., & Manichand, D. (2022) · Maandblad voor Accountancy en Bedrijfseconomie · doi:10.5117/mab.96.90108 · Cited by 12
The Development Landscape of Large Language Models for Biomedical Applications
Cao, Z., Keloth, V. K., Xie, Q., Qian, L., Liu, Y., Wang, Y., Shi, R., Zhou, W., Yang, G. F., Zhang, J., et al. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-102224-074736 · Cited by 21
Li, Y., & Goel, S. (2025) · Advances in Accounting · doi:10.1016/j.adiac.2025.100842 · Cited by 27
Navigating the future of artificial intelligence technologies for improving the care of older adults
Abadir, M., Dineen, W., Myers, D., Yu, S., & Phan, P. (2025) · Innovation in Aging · doi:10.1093/geroni/igaf092 · Cited by 15
Disentangling the Components of Ethical Research in Machine Learning
Ashurst, C., Barocas, S., Campbell, R., & Raji, I. D. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533781 · Cited by 29
“Mandating the use of standardize reporting practices such as datasheets [27], data nutrition labels [64] and modelcards [46], can also ensure disclosure of limitations etc.”
BackgroundStakeholder-accountability model for artificial intelligence projects
Maxmetrics, H., & Miller, G. J. (2022) · Journal of Economics and Management · doi:10.22367/jem.2022.44.18 · Cited by 24
Meiser, M., & Zinnikus, I. (2024) · Energies · doi:10.3390/en17091992 · Cited by 21
Lazy Data Practices Harm Fairness Research
Simson, J., Fabris, A., & Kern, C. A. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658931 · Cited by 34
“A datasheet [48] for the COMPAS dataset is available in the Appendix of Fabris et al. [42].”
MethodMachine learning data practices through a data curation lens: An evaluation framework
Bhardwaj, E., Gujral, H., Wu, S., Zogheib, C., Maharaj, T., & Becker, C. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658955 · Cited by 44
“The framework builds on the significant impact of datasheets [42] and takes the logical next step.”
BackgroundMethodInfluentialA comprehensive review of techniques for documenting artificial intelligence
Königstorfer, F. (2024) · Digital Policy Regulation and Governance · doi:10.1108/dprg-01-2024-0008 · Cited by 18
Constructing Capabilities: The Politics of Testing Infrastructures for Generative AI
Grill, G. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659009 · Cited by 21
“Documenting expected causal relations in publications, and other contextual information relevant to understanding algorithmic systems, can also aid researchers in judging the quality of claims made by, and about, these systems [50, 78].”
BackgroundBuilding Guardrails in AI Systems with Threat Modeling
Dev, J., Akhuseyinoglu, N. B., Kayas, G., Rashidi, B., & Garg, V. (2024) · Digital Government Research and Practice · doi:10.1145/3674845 · Cited by 18
“— Confidentiality attacks to ML training data [17]: This attack type refers to disclosure of sensitive data in the training set, such as those related to health, finance, or protected by law.”
Nullius in Explanans: an ethical risk assessment for explainable AI
Nannini, L., Huyskes, D., Panai, E., Pistilli, G., & Tartaro, A. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09800-7 · Cited by 18
“Finally, to support the practical application of our framework, we aim to develop standardized tools and metrics for XAI risk assessment, similar to the efforts made by (Arnold et al., 2019; Gebru et al., 2021; Mitchell et al., 2018; Sokol & Flach, 2020).”
The Venus score for the assessment of the quality and trustworthiness of biomedical datasets
Chicco, D., Fabris, A., & Jurman, G. (2025) · BioData Mining · doi:10.1186/s13040-024-00412-x · Cited by 18
“The former was released in 2017, and the latter in 1989, showing that good practices of data documentation can be independent of dataset age, despite growing awareness about this topic in recent years [11, 64].”
InfluentialNavigating AI conformity: A design framework to assess fairness, explainability, and performance
von Zahn, M., Zacharias, J., Lowin, M., Chen, J., & Hinz, O. F. (2025) · Electronic Markets · doi:10.1007/s12525-025-00770-2 · Cited by 22
“Therefore, inspired by existing works aiming to measure the explainability of AI systems (see, e.g., Gebru et al., 2021; Mitchell et al., 2019), we develop a questionnaire that the AI owner fills out as part of the conformity assessment.”
AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development
Saxena, D., Jung, J.-Y., Forlizzi, J., Holstein, K., & Zimmerman, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714098 · Cited by 34
“These approaches include fairness toolkits, auditing meth-ods, algorithmic “de-biasing” techniques, and documentation protocols [29, 32, 41, 51, 58, 79, 80, 120].”
Pneuma : Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System
Balaka, M. I. L., Alexander, D., Wang, Q., Gong, Y., Krisnadhi, A. A., & Fernandez, R. C. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725337 · Cited by 54
What Machine Learning Can Learn From Software Modularity
Kriens, P., & Verbelen, T. (2022) · Computer · doi:10.48550/arxiv.1906.10366 · Cited by 18
“Meanwhile, other scientists also stress the importance of more carefully cu-rating metadata for datasets [18] and many other (hyper)parameters involved in ML model training [19].”
BackgroundDeo, S., & Sontakke, N. (2021) · Computer · doi:10.1109/mc.2021.3076851 · Cited by 16
School Choice Algorithms: Data Infrastructures, Automation, and Inequality
Swist, T., & Gulson, K. Ν. (2022) · Postdigital Science and Education · doi:10.1007/s42438-022-00334-z · Cited by 23
Jin, Z., Wang, X., Cheng, F., Sun, C., Liu, Q., & Qu, H. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3236380 · Cited by 12
“For example, clear and detailed documentation of benchmark datasets should be carefully considered [22], [23].”
BackgroundMLTEing Models: Negotiating, Evaluating, and Documenting Model and System Qualities
Maffey, K. R., Dotterrer, K., Niemann, J., Cruickshank, I. J., Lewis, G. A., & Kästner, C. (2023) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier58687.2023.00012 · Cited by 13
InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations
Feldhus, N., Wang, Q., Anikina, T., Chopra, S., Oguz, C., & Möller, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.359 · Cited by 15
Chavan, V., Koch, P., Schlüter, M., & Briese, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01057 · Cited by 27
“…related metadata (weight, length, breadth, and height of objects, along with the superclass, material, shape, colour and additional descriptors) and a datasheet [24] are available on the online repository. object placed along with other objects from the lab in the background and no occlusion.”
BackgroundAssessing Quality of Surgical Real-World Data from an Automated Electronic Health Record Pipeline
Corey, K., Helmkamp, J. K., Simons, M. G., Curtis, L. H., Marsolo, K. A., Balu, S., Gao, M. C., Nichols, M., Watson, J. A., Mureebe, L., et al. (2020) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2019.12.005 · Cited by 13
AI and Holistic Review: Informing Human Reading in College AdmissionsPreprint
Alvero, A., Arthurs, N., antonio, A. L., Domingue, B. W., Gebre-Medhin, B., Giebel, S., & Stevens, M. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09318 · Cited by 12
Customizing Triggers with Concealed Data Poisoning.Preprint
Wallace, E., Zhao, T. Z., Feng, S., & Singh, S. (2020) · arXiv (Cornell University) · Cited by 12
Pitfalls in Machine Learning Research: Reexamining the Development CyclePreprint
Biderman, S., & Scheirer, W. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02832 · Cited by 25
“From the point of view of documentation, the “Datasheets for Datasets” [21] framework is something that people are beginning to use and which would benefit the field if adopted as a standard [43, 15, 47].”
Societal Biases in Language Generation: Progress and ChallengesPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04054 · Cited by 12
“Paullada et al. (2020), Bender and Fried-man (2018), and Gebru et al. (2018) provide more comprehensive surveys and frameworks that focus on aspects of data creation and management that could lead to biases, and we refer readers to their works for more discussion.”
BackgroundBenchmarking Multimodal AutoML for Tabular Data with Text FieldsPreprint
Shi, X., Mueller, J. W., Erickson, N., Li, M., & Smola, A. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02705 · Cited by 52
Trustworthy AI for safe medicines
Stegmann, J., Littlebury, R., Trengove, M., Goetz, L., Bate, A., & Branson, K. (2023) · Nature Reviews Drug Discovery · doi:10.1038/s41573-023-00769-4 · Cited by 13
Daneshjou, R., & Kittler, H. (2023) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2023.11.011 · Cited by 12
Ten simple rules for building and maintaining a responsible data science workflow
Stoudt, S., Jernite, Y., Marshall, B., Marwick, B., Sharan, M., Whitaker, K., & Danchev, V. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1012232 · Cited by 11
AI Model Passport: Data and system traceability framework for transparent AI in health
Kalokyri, V., Tachos, N. S., Kalantzopoulos, C., Sfakianakis, S. G., Kondylakis, H., Zaridis, D. I., Colantonio, S., Regge, D., Papanikolaou, N. K., Kostas Marias, A., et al. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.09.041 · Cited by 11
“While initiatives like Datasheets for Datasets [9], Model Cards [10], and Google Data Cards [11] promote structured documentation, they typically rely on manual, human-readable templates completed after development, and lack automation, standardization, and machine interpretability, limiting…”
Vo, T. H., & Le, N. Q. K. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag227 · Cited by 15
“Established reporting frameworks such as model cards and datasheet-style dataset documentation provide structured mechanisms for disclosing such information and enhancing auditability [76, 77].”
Exploring Social Biases of Large Language Models in a College Artificial Intelligence Course
Kolisko, S., & Anderson, C. J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26879 · Cited by 25
“The in-class content included a taxonomy of potential harms (Blodgett 2021); a discussion of existing bias probe tasks; a lecture on reproducibility and model/data documentation (Mitchell et al. 2019; Gebru et al. 2021); small group discussions about the ethics of developing AI technology in various scenarios; and a l…”
Ilager, S., Toosi, A. N., Jha, M. R., Brandić, I., & Buyya, R. (2023) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1145/3603166.3632137 · Cited by 13
Gausen, A., Mitra, B., & Lindley, S. E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658900 · Cited by 21
“Gebru et al. described them as having "unfathomable training data" [53].”
BackgroundUnderstanding Disparities in Post Hoc Machine Learning Explanation
Mhasawade, V., Rahman, S., Haskell-Craig, Z., & Chunara, R. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659043 · Cited by 19
“While efforts to-date in explanation methods have predominantly focused on improving the post hoc methods [6, 18], this overlooks the role of data and black box models in generating unfair explanations.”
MethodUmmenhofer, B., Agrawal, S., Sepulveda, R., Lao, Y., Zhang, K., Cheng, T., Richter, S. R., Wang, S., & Ros, G. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00097 · Cited by 27
Tran, M. H., & Fiesler, C. L. (2024) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3632620.3671109 · Cited by 16
Jiang, S., Ashar, P., Shandhi, M. M. H., & Dunn, J. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00170-5 · Cited by 13
Pelekis, S., Karakolis, E., Lampropoulos, G., Mouzakitis, S., Markaki, O. I., Ntanos, C., & Askounis, D. (2024) · IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC) · doi:10.1109/ice/itmc61926.2024.10794222 · Cited by 22
“Usage of methodologies to ensure transparency of data management processes and their usefulness for users (e.g Datasheets for Datasets [58])”
How can we make sound replication decisions?
Davis‐Stober, C. P., Sarafoglou, A., Aczél, B., Chandramouli, S., Errington, T. M., Field, S. M., Fishbach, A., Freire, J., Ioannidis, J. P. A., Oberauer, K., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401236121 · Cited by 15
Kwao, L., Yang, Y., Zou, J., & Ma, J. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70001 · Cited by 26
“Although biases in AI models can lead to unfair or incorrect detection of rumors, concerns about transparency and privacy are central to maintaining user trust and adhering to ethical standards (Calo 2017; Diakopoulos 2016; Dignum 2019; Gebru et al. 2021).”
InfluentialFahad, S. K. A., Zhengkui, D. W., Ng, P. C., Wong, N., Ng, A. B., & See, S. C. W. (2025) · Studies in computational intelligence · doi:10.1007/978-3-031-80334-5_19 · Cited by 13
Towards Fairness in AI for Melanoma Detection: Systemic Review and Recommendations
Montoya, L. N., Roberts, J. S., & Hidalgo, B. S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-84460-7_21 · Cited by 22
Promoting transparency in AI for biomedical and behavioral research
Hernandez‐Boussard, T., Lee, A., Stoyanovich, J., & Biven, L. (2025) · Nature Medicine · doi:10.1038/s41591-025-03680-0 · Cited by 11
Leashes, not guardrails: A management‐based approach to artificial intelligence risk regulation
Coglianese, C., & Crum, C. R. (2025) · Risk Analysis · doi:10.1111/risa.70020 · Cited by 21
“…attempts to systematize information collection and reporting have taken the form of “model cards” (Mitchell et al., 2019)—also called Datasheets (Gebru et al., 2021) or FactSheets (Richards et al., 2020)— which contain basic information about an AI tool, such as the model type and dataset used.”
Mahajan, K., & Shaikh, S. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.36 · Cited by 32
“This (potentially non-exhaustive) list of questions is inspired by the current movement in several research fields to pre-register studies in advance (Nosek et al., 2019; Vilhuber, 2020) and the needs for datasheets for datasets (Gebru et al., 2018).”
BackgroundOmena, J. J., Pilipets, E., Universität Klagenfurt, D. D. M. Y. C., Gobbo, B., Jason, C., & Universität Siegen, C. C. D. I. “. D. C. (2021) · Revista Diseña · doi:10.7764/disena.19.article.1 · Cited by 11
Reflections and Considerations on Running Creative Visualization Learning Activities
Roberts, J. C., Bach, B., Boucher, M., Chevalier, F., Diehl, A., Hinrichs, U., Huron, S., Kirk, A., Knudsen, S., Meirelles, I., et al. (2022) · HAL · Cited by 11
Trust and Transparency in Contact Tracing ApplicationsPreprint
Hobson, S. F., Hind, M., Mojsilović, A., & Varshney, K. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11356 · Cited by 15
“Thus, there has been a recent focus on transparent reporting mechanisms for AI systems, focusing on datasets [5], [6], [7], models [8], [9] and services [8].”
BackgroundRepresentation Matters: Assessing the Importance of Subgroup Allocations in Training DataPreprint
Rolf, E., Worledge, T., Recht, B., & Jordan, M. I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03399 · Cited by 25
“dition to inclusivity, transparency and reusability (Gebru et al., 2018; Gebru, 2020; Wilkinson et al., 2016).”
BackgroundAdversarial Machine Learning and Cybersecurity
Musser, M. (2023) · doi:10.51593/2022ca003 · Cited by 20
Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits
Gerchick, M. K., Encarnación, R., Tanigawa-Lau, C., Armstrong, L., Gutiérrez, A., & Metaxa, D. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732004 · Cited by 10
“For instance, auditors could be required to include datasheets [41] or model cards [90] about audit data and employer-set thresholds.”
Ramos, F. (2025) · Laws · doi:10.3390/laws14060084 · Cited by 22
“Consequently, the integration of metadata and logging infrastructures (Gebru et al. 2021; Mitchell et al. 2019) provides a systematic means of documenting technical configurations, user interventions, and version histories, thereby constituting an audit framework essential for both regulatory and…”
Research Reproducibility as a Survival Analysis
Raff, E. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16124 · Cited by 10
Socially meaningful transparency in data-based systems: reflections and proposals from practice
Bates, J., Kennedy, H., Medina-Perea, I., Oman, S., & Pinney, L. (2023) · Journal of Documentation · doi:10.1108/jd-01-2023-0006 · Cited by 19
A Siren Song of Open Source Reproducibility, Examples from Machine Learning
Raff, E., & Farris, A. (2023) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3589806.3600042 · Cited by 23
Heyn, H.-M., Habibullah, K. M., Knauss, E., Horkoff, J., Borg, M., Knauss, A., & Li, P. J. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00011 · Cited by 23
“Related work: [12], [20] Data-II Accept that a data specification can only be created iteratively.”
BackgroundCollect, Measure, Repeat: Reliability Factors for Responsible AI Data Collection
Inel, O., Draws, T., & Aroyo, L. M. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27547 · Cited by 25
Metadata
- Authors
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K.
- Affiliations
- DAIR Institute (US)
- University of Washington (US)
- Cornell University (US)
- Microsoft Research New York City (United States) (US)
- University of Maryland, College Park, MD
- University of Southern California (US)
- ORCID
- Venue
- Communications of the ACM
- Published
- 2021-11-19
- Volume
- 64
- Issue
- 12
- Pages
- 86-92
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2795038878
- Semantic Scholar
- 0df347f5e3118fac7c351917e3a497899b071d1e
- Cited by
- 2,469
- References
- 37
- Citations by year
- 2026: 788 · 2025: 402 · 2024: 298 · 2023: 379 · 2022: 221 · 2021: 217
- FWCI
- 252.30
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Alfred P. Sloan Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Keywords
- dataset documentation, data provenance, data sharing, data reuse, metadata
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.