Data Statements: From Technical Concept to Community Practice
Open accessAbstract
Responsible computing ultimately requires that technical communities develop and adopt tools, processes, and practices that mitigate harms and support human flourishing. Prior efforts toward the responsible development and use of datasets, machine learning models, and other technical systems have led to the creation of documentation toolkits to facilitate transparency, diagnosis, and inclusion. This work takes the next step: to catalyze community uptake, alongside toolkit improvement. Specifically, starting from one such proposed toolkit specialized for language datasets, data statements for natural language processing, we explore how to improve the toolkit in three senses: (1) the content of the toolkit itself, (2) engagement with professional practice, and (3) moving from a conceptual proposal to a tested schema that the intended community of use may readily adopt. To achieve these goals, we first conducted a workshop with natural language processing practitioners to identify gaps and limitations of the toolkit as well as to develop best practices for writing data statements, yielding an interim improved toolkit. Then we conducted an analytic comparison between the interim toolkit and another documentation toolkit, datasheets for datasets. Based on these two integrated processes, we present our revised Version 2 schema and best practices in a guide for writing data statements. Our findings more generally provide integrated processes for co-evolving both technology and practice to address ethical concerns within situated technical communities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data sheets, model cards, and system cards are frameworks that can help structure the systematic presentation of metadata about models and systems and that feed into auditing procedures.
“Important constituent elements are the notions of data sheets [18, 25, 38], model cards [40], and system cards [22]: frameworks that can help structure the systematic presentation of metadata about models and systems and that feed into auditing procedures [37].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [38]
Section 2.1 describes documentation toolkits as structured ways to provide information about datasets, models, and AI systems, and says they are being integrated into standards for managing bias. The cited paper does not establish that these frameworks feed into auditing procedures; it also does not discuss system cards specifically. No page number was available in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measuredData sheets document key aspects of data collection and curation and ensure that relevant information about training data is made available in systematic and relatively standardised ways.
“Data sheets document key aspects of data collection and curation [18, 38]. They ensure that relevant information about training data is made available in systematic and relatively standardised ways.”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1× · see citing works
“In their initial work, Bender and Friedman began with a conceptual investigation, drawing on the definition of bias presented in an article by Friedman and Nissenbaum [11] as “systematic” and “unfair discrimination.”
BackgroundSelf-citationFriedman, B., & Hendry, D. G. (2019) · The MIT Press eBooks · doi:10.7551/mitpress/7585.001.0001 · Cited by 960 · see citing works
Self-citationBender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · cited 9× · see citing works
“…Bender and Friedman as “a characterization of a dataset that provides context to allow developers and users to better understand how experimental results might generalize, how software might be appropriately deployed, and what biases might be reflected in systems built on the software” [2, p. 587].”
BackgroundMethodResultInfluentialSelf-citationDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · see citing works
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
Directions in abusive language training data, a systematic review: Garbage in, garbage out
Vidgen, B., & Derczynski, L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243300 · Cited by 335 · see citing works
The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
Holland, S., Hosny, A., Newman, S., Joseph, J., & Chmielinski, K. (2018) · arXiv · doi:10.48550/arxiv.1805.03677 · Cited by 175 · see citing works
Informed consent in the Mozilla browser: implementing value-sensitive design
Friedman, B., Howe, D. C., & Felten, E. W. (2003) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2002.994366 · Cited by 140 · see citing works
Self-citationCookies and Web browser design
Millett, L. I., Friedman, B., & Felten, E. W. (2001) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365034 · Cited by 122 · see citing works
Self-citationNathan, L. P., Klasnja, P., & Friedman, B. (2007) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1240866.1241046 · Cited by 105 · see citing works
Self-citationDatasheets for Datasets help ML Engineers Notice and Understand Ethical Issues in Training Data
Boyd, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479582 · Cited by 67 · see citing works
Value sensitive design as a formative framework
Hendry, D. G., Friedman, B., & Ballard, S. A. (2021) · Ethics and Information Technology · doi:10.1007/s10676-021-09579-x · Cited by 52 · see citing works
Self-citationIEEE Standard for Transparency of Autonomous Systems
Winfield, A., Hossaini, A., Rajabiyazdi, F., Rannow, R. K., Theodorou, A., Underwood, M. A., von Stryk, O., Wortham, R. H., Watson, E., Egawa, T., et al. (2021) · doi:10.1109/ieeestd.2022.9726144 · Cited by 52 · see citing works
Designing for energy transition through Value Sensitive Design
Mok, L., & Hyysalo, S. (2017) · Design Studies · doi:10.1016/j.destud.2017.09.006 · Cited by 50 · see citing works
Development of a Privacy Addendum for Open Source Licenses: Value Sensitive Design in Industry
Friedman, B., Smith, I., Kahn, P. H., Consolvo, S., & Selawski, J. (2006) · Lecture notes in computer science · doi:10.1007/11853565_12 · Cited by 42 · see citing works
Self-citationNutritional Labels for Data and Models
Stoyanovich, J., & Howe, B. (2019) · IEEE Data(base) Engineering Bulletin · Cited by 36 · see citing works
A Methodology for Creating AI FactSheets
Richards, J., Piorkowski, D., Hind, M., Houde, S., & Mojsilović, A. (2020) · arXiv · doi:10.48550/arxiv.2006.13796 · Cited by 28 · see citing works
Chmielinski, K., Newman, S., Taylor, M., Joseph, J., Thomas, K., Yurkofsky, J., & Qiu, Y. C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.03954 · Cited by 26 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Juliet M. Corbin and Anselm L. Strauss. 2008. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (3rd ed.), Juliet Corbin and Anselm Strauss (Eds.). SAGE, Los Angeles, CA.
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna M. Wallach, Hal Daumé III, and Kate Crawford. 2020. Datasheets for datasets. CoRR abs/1803.09010v7 (2020). · arXiv
- Lassana Magassa and Batya Friedman. [n.d]. Toward inclusive justice: Applying the diverse voices design method to improve the Washington State access to justice technology principles. Under review.
- Safiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press, New York, NY.
Cited by
Sorted by citations
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111 · see citing works
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 67 · see citing works
Improving Governance Outcomes Through AI Documentation: Bridging Theory and Practice
Winecoff, A., & Bogen, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713814 · Cited by 16 · see citing works
Identification and Description of Emotions by Current Large Language ModelsPreprint
Patel, S., & Fan, J. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.17.549421 · Cited by 17 · see citing works
Developing responsible AI practices at the Smithsonian Institution
Dikow, R. B., DiPietro, C., Trizna, M. G., BredenbeckCorp, H., Bursell, M. G., Ekwealor, J. T. B., Hodel, R. G. J., Lopez, N., Mattingly, W., Munro, J., et al. (2023) · Research Ideas and Outcomes · doi:10.3897/rio.9.e113334 · Cited by 6 · see citing works
A Semantic Approach for Linked Model, Data, and Dataspace Cards
Donald, A., Galanopoulos, A., Curry, E., Muñoz, E., Ullah, I., Waskow, M. A., Kalra, M., Saxena, S., & Iqbal, T. (2025) · IEEE Access · doi:10.1109/access.2025.3572211 · Cited by 3 · see citing works
What is a Data Document? Analyzing Four Emerging Data Documentation Frameworks in AI / ML
Maemura, E. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1270 · Cited by 3 · see citing works
Explainable AI for Hate Speech Moderation: A Stakeholder‐Centered and Sociotechnical Review
Qureshi, M. D. M., Qureshi, M. A., & Rashwan, W. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70076 · Cited by 6 · see citing works
Letterie, G. S. (2024) · Human Reproduction · doi:10.1093/humrep/deae144 · Cited by 3 · see citing works
Collaborative Growth: When Large Language Models Meet Sociolinguistics
Nguyen, D. (2025) · Language and Linguistics Compass · doi:10.1111/lnc3.70010 · Cited by 5 · see citing works
Scantamburlo, T., & Zollo, F. (2026) · ACM Transactions on the Web · doi:10.1145/3796546 · Cited by 2 · see citing works
“What data might be out of date? All of it!”: web archivist perspectives on datasheets for datasets
Maemura, E. (2026) · Journal of Documentation · doi:10.1108/jd-05-2026-0270 · Cited by 1 · see citing works
Evaluation of AI ethics tools in language models: a developers' perspective case study
Silva, J., Moreira, D. A. B., dos Santos, G. D., Ferreira, A. I., Maia, H. A., Avila, S., & Pedrini, H. (2025) · AI and Ethics · doi:10.1007/s43681-025-00914-2
Bottomly, D., Suciu, C. G., Cordier, B. A., Evans, N., Poire, A., ZHENG, C. L., Myers, J. L., Sandulache, V. C., Bivona, T. G., Roth, J., et al. (2026) · Scientific Data · doi:10.1038/s41597-026-06670-0 · Cited by 1 · see citing works
Zhang, A., & Lee, M. K. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791306
Scheuerman, M. K., Hutiri, W. T., Rahmattalabi, A., Matthews, V., Xiang, A., & Andrews, J. T. A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790364
How to (Un)Link the Bias: Linked Data to Counteract Dataset Bias
Cimitan, A., Alves-Pinto, A., & Geierhos, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-25308-8_8
Rothschild, A., Nematova, M. K., Eickman, B., DiSalvo, C. F., & DiSalvo, B. J. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808052
Bureaucratic Silences: What the Canadian AI Register Reveals, Omits, and Obscures
Das, D., Tessono, C., Ahmed, S. I., & Guha, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812336 · Cited by 2 · see citing works
Maemura, E., & Byrne, H. (2026) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.70113
Berendsen, S., & Beauxis-Aussalet, E. (2026) · AI and Ethics · doi:10.1007/s43681-026-01375-x
- Authors
- McMillan-Major, A., Bender, E. M., & Friedman, B.
- Affiliations
- University of Washington (US)
- Venue
- ACM Journal on Responsible Computing
- Published
- 2024-03-20
- Volume
- 1
- Issue
- 1
- Pages
- 1-17
- ISSN
- 2832-0565 (ISSN-L)
- OpenAlex
- W4375861770
- Cited by
- 21
- References
- 24
- Citations by year
- 2026: 12 · 2025: 4 · 2024: 2 · 2023: 3
- FWCI
- 5.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Journal on Responsible Computing (publishedVersion, cc-by)
- Funders
- Tech Policy Lab
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- responsible computing, natural language processing, best practices, transparency, eudaimonia, technical communities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
