Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure
Open accessThe DOI matches its Crossref record; 143 of the 143 references it cites were found in a registry or an index.
Abstract
Datasets that power machine learning are often used, shared, and reused with little visibility into the processes of deliberation that led to their creation. As artificial intelligence systems are increasingly used in high-stakes tasks, system development and deployment practices must be adapted to address the very real consequences of how model development data is constructed and used in practice. This includes greater transparency about data, and accountability for decisions made when developing it. In this paper, we introduce a rigorous framework for dataset development transparency that supports decision-making and accountability. The framework uses the cyclical, infrastructural and engineering nature of dataset development to draw on best practices from the software development lifecycle. Each stage of the data development lifecycle yields documents that facilitate improved communication and decision-making, as well as drawing attention to the value and necessity of careful data work. The proposed framework makes visible the often overlooked work and decisions that go into dataset creation, a critical step in closing the accountability gap in artificial intelligence and a critical/necessary resource aligned with recent work on auditing processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Important constituent elements are the notions of data sheets [18, 25, 38], model cards [40], and system cards [22]: frameworks that can help structure the systematic presentation of metadata about models and systems and that feed into auditing procedures [37].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Work on AI safety and reproducibility has long pointed out the crucial importance of training data for understanding model performance, ensuring reproducibility, and assessing legal exposure [2, 25, 29, 46, 58].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This has been flagged as one of the causes of historical and representational bias in ML systems that use those data, with numerous authors urging for slower and more methodical approaches to data collection [36,53].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Earlier trends of opportunistic data collection and post hoc justifications of large datasets are gradually moving towards a requirement for more deliberative data collection methods [35,36], sampling techniques [16] and minimal data architectures [57] to deliver better performance without reducing model accuracy.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Numerous authors have observed that ML systems perform better for users whose demographic characteristics match those represented in the training data [9,36].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [36]
Experimental — accuracy not yet measured
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dilemmas in a general theory of planning
Rittel, H. W. J., & Webber, M. M. (1973) · Policy Sciences · doi:10.1007/bf01405730 · Cited by 16,338 · cited 2×
“s. Undesirable data feedback loops and unutilized data features can be mitigated with careful requirements practices. Data Postmortems and Premortems. Even the best designed systems occasionally fail [25, 119], however cyclical development practiceshelpavoidmakingthesamefailurestwice,andthepostmortem is a critical too…”
BackgroundInfluentialScience in Action: How to Follow Scientists and Engineers through Society
Bijker, W. E., & Latour, B. (1988) · Technology and Culture · doi:10.2307/3105094 · Cited by 10,767
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
(1987) · Cited by 8,955
Laboratory Life: The Construction of Scientific Facts
Latour, B. (1986) · DigitalGeorgetown (Georgetown University Library) · Cited by 7,491
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · PhilPapers (PhilPapers Foundation) · Cited by 6,992
Representing and Intervening: Introductory Topics in the Philosophy of Natural Science
Hacking, I. (1983) · doi:10.1017/cbo9780511814563 · Cited by 4,814
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567 · cited 1×
“how consent is obtained, and the data collection process needs to keep “the receipts” as evidence of due process. To facilitate this, it is therefore necessary that dataset requirements be traceable [100], and in many cases requirements should be measurable. Non-boolean test outcomes are still useful in flagging conce…”
BackgroundThe Politics and Poetics of Infrastructure
Larkin, B. (2013) · Annual Review of Anthropology · doi:10.1146/annurev-anthro-092412-155522 · Cited by 4,095 · cited 1×
“s, with which they interact through technical interfaces. Datasets are also infrastructure in the infrastructure studies sense, in that they arrange a way to view and structure knowledge of the world [22, 88]. We show how framing datasets as infrastructure helps to shed light on many of the social and technical phenom…”
BackgroundWicked Problems in Design Thinking
Buchanan, R. (1992) · Design Issues · doi:10.2307/1511637 · Cited by 3,539 · cited 1×
“s. Undesirable data feedback loops and unutilized data features can be mitigated with careful requirements practices. Data Postmortems and Premortems. Even the best designed systems occasionally fail [25, 119], however cyclical development practiceshelpavoidmakingthesamefailurestwice,andthepostmortem is a critical too…”
BackgroundA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Self-citationThe structure of ill structured problems
Simon, H. A. (1973) · Artificial Intelligence · doi:10.1016/0004-3702(73)90011-8 · Cited by 2,600 · cited 1×
“[26], and the infrastructural properties of data discussed above suggest the same is true of ML dataset workers. 3.4 Dataset development is non-linear Engineering problems “tend to be ill-structured” [133] or even “wicked” [119], meaning they defy deductive reasoning and definitive answers. Instead, engineering involv…”
BackgroundInfluentialRevisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Winner, L. (2017) · Computer Ethics · doi:10.4324/9781315259697-21 · Cited by 2,317
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 3×
“or opportunistically. From an observer’s perspective, these practices replace abductive and a posteriori reasoning about dataset provenance with careful documentation, facilitating reviews and audits [114]. They help us to avoid the trap of being blindly “data-driven” by poorly-understood datasets— mistaking taking a…”
BackgroundInfluentialSelf-citationThe Unreasonable Effectiveness of Data
Halevy, A., Norvig, P., & Pereira, F. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,767
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Representing and Intervening: Introductory Topics in the Philosophy of Natural Science.
Baird, D., & Hacking, I. (1988) · Noûs · doi:10.2307/2215864 · Cited by 1,584
Representativeness in Corpus Design
BIBER, D. (1993) · Literary and Linguistic Computing · doi:10.1093/llc/8.4.243 · Cited by 1,269
Research dilemmas: Paradigms, methods and methodology
Mackenzie, N., & Knipe, S. (2006) · Issues in educational research · Cited by 1,208
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 1×
“icism in recent years due to their tendency to perpetuate and amplify social inequality [13, 44]. Data is frequently identified as a key source of these failures through its role in “bias-laundering” [40, 51, 54, 118, 124]. For example, recent studies have uncovered widespread prevalence of undesirable biases in machi…”
BackgroundDevelop Long-Term Competitiveness through IT Assets
Ross, J. W., Beath, C. M., & Goodhue, D. L. (1996) · Cited by 1,056
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
Roh, Y., Heo, G., & Whang, S. E. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2946162 · Cited by 968
Introduction to the Special Issue on the Web as Corpus
Kilgarriff, A., & Grefenstette, G. (2003) · Computational Linguistics · doi:10.1162/089120103322711569 · Cited by 929
(2015) · neural information processing systems · Cited by 912
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · cited 1×
“right”. Indeed, premortems can be a useful tool during data requirements analysis for identifying gaps and vagueness. Transparent Dataset Reporting. Transparent reporting of machine learning datasets [16, 52, 70] and the models trained on those data [99] enables third party agents, including potential users, to make i…”
MethodWikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829
Turkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798 · cited 1×
“ring for datasets (including the collection, cleaning, and annotation of data) receives few, if any, accolades in the mainstream field. Since it is often an unpaid burden on top of existing workloads [73, 124, 138], careful dataset work is unlikely to occur at all. Publications focusing solely on datasets rarely make…”
Background“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“for cultural shifts (Section 5). 2 DATASETS AS INFRASTRUCTURE ML fairness practitioners have identified collection and curation of representative data as the most common way to mitigate model biases [71], echoing the broader tendency within ML communities to valorize larger datasets as necessarily better—exemplified b…”
MethodLooking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1023 · Cited by 683
Responsibility from the Margins
Shoemaker, D. (2015) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198715672.001.0001 · Cited by 653
Simulation and similarity: using models to understand the world
(2013) · Choice Reviews Online · doi:10.5860/choice.51-0251 · Cited by 630
I.—Knowing How and Knowing that: The Presidential Address
Ryle, G. (1946) · Proceedings of the Aristotelian Society · doi:10.1093/aristotelian/46.1.1 · Cited by 617
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C. J. C., & Renshaw, E. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 610
Software maintenance and evolution: a roadmap
Bennett, K. H., & Rajlich, V. T. (2000) · Proceedings of the Conference on The Future of Software Engineering · doi:10.1145/336512.336534 · Cited by 606
Data reuse and the open data citation advantage
Piwowar, H. A., & Vision, T. J. (2013) · PeerJ · doi:10.7717/peerj.175 · Cited by 603
Fine-grained recognition without part annotations
Krause, J., Jin, H., Yang, J., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299194 · Cited by 466
Attributability, Answerability, and Accountability: Toward a Wider Theory of Moral Responsibility
Shoemaker, D. (2011) · Ethics · doi:10.1086/659003 · Cited by 454
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434
Interpretability of deep learning models: A survey of results
Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., Srivastava, M., Preece, A., Julier, S., Rao, R. M., et al. (2017) · 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397411 · Cited by 422
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414
Patton, R. (2000) · Cited by 369
TFX: A TensorFlow-Based Production-Scale Machine Learning Platform
Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., et al. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098021 · Cited by 350
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · cited 3×
“are all directly applicable to datasets. Table 2 summarizes how her concerns relate to datasets, as well as the mitigating proposals which we will elaborate below.1 Our analysis and proposals follow [145] and [83] in using temporal situatedness as a key organizing factor. At the highest level, what are the dataset acc…”
BackgroundMethodInfluentialThe ML test score: A rubric for ML production readiness and technical debt reduction
Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017) · 2017 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2017.8258038 · Cited by 310
Infrastructure: The Social Value of Shared Resources
Frischmann, B. M. (2012) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199895656.001.0001 · Cited by 297
Problem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283 · cited 2×
“defy deductive reasoning and definitive answers. Instead, engineering involves reconceptualizing a complex situation to facilitate analysis; with feedback loops impacting problem (re-)definition (see [109]), rather than focusing narrowly on problem solution. As goal-oriented models of reality, dataset development requ…”
BackgroundMethodSocial Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.487 · Cited by 274
Self-citationSocial Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · arXiv · doi:10.48550/arxiv.2005.00813 · Cited by 274 · cited 1×
“udies have uncovered widespread prevalence of undesirable biases in machine learning datasets, such as the under-representation of minoritized groups [27, 40, 130] and stereotype aligned correlations [28, 51, 72, 153]. Datasets also frequently reflect historical patterns of social injustices, which can subsequently be…”
BackgroundSelf-citationDiscussion of the Method : Conducting the Engineer's Approach to Problem Solving
Koen, B. V. (2003) · Cited by 267
An Introduction to the Dataverse Network as an Infrastructure for Data Sharing
King, G. (2007) · Sociological Methods & Research · doi:10.1177/0049124107306660 · Cited by 262
Large image datasets: A pyrrhic win for computer vision?
Birhane, A., & Prabhu, V. U. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00158 · Cited by 257
Loshin, D. S. (2008) · Cited by 255
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240
Coveney, P. V., Dougherty, E. R., & Highfield, R. R. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2016.0153 · Cited by 229 · cited 1×
“to the development lifecycle. Seen through this lens, dataset development requires a great deal of further dedicated research (e.g., on practices of collection, methods and protocols), its own theory [35],6 conferences, prizes, etc. There is a need to shift the perception of dataset work away from crafty gluework to h…”
BackgroundMaking Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research
Vaughan, J. W. (2018) · Journal of Machine Learning Research · Cited by 227
On the Reuse of Scientific Data
Pasquetto, I. V., Randles, B. M., & Borgman, C. L. (2017) · Data Science Journal · doi:10.5334/dsj-2017-008 · Cited by 223
Requirements Engineering for Machine Learning: Perspectives from Data Scientists
Vogelsang, A., & Borg, M. (2019) · 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew.2019.00050 · Cited by 223
Data Lifecycle Challenges in Production Machine Learning: A Survey
Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2018) · ACM SIGMOD Record · doi:10.1145/3299887.3299891 · Cited by 214 · cited 1×
“cle stages is sometimes debated, for the current purposes it 1Nissenbaum’s fourth concern is about liability, which is beyond the scope of this paper. 2We differ from the “data lifecycle” analysis of [112] in that they consider the “first step of ML is to understand your data”, presuming that data already exists. , ,…”
BackgroundPeacocke, C., & Nagel, T. (1988) · Philosophy and Phenomenological Research · doi:10.2307/2108026 · Cited by 198
CoCoA: A General Framework for Communication-Efficient Distributed Optimization
Smith, V., Forte, S., Ma, C., Takáč, M., Jordan, M. I., & Jaggi, M. (2016) · arXiv (Cornell University) · doi:10.3929/ethz-b-000282738 · Cited by 193
Scheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392866 · Cited by 193 · cited 1×
“and standards of data collection within ML have themselves been subject to critique, with scholars identifying insufficient documentation and transparency regarding processes of dataset construction [52, 53, 125], as well as problematic consent practices [113]. The lack of accountability to datafied and surveilled pop…”
BackgroundPerforming a Project Premortem
Klein, G. (2008) · IEEE Engineering Management Review · doi:10.1109/emr.2008.4534313 · Cited by 183
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Self-citationThe Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
Holland, S., Hosny, A., Newman, S., Joseph, J., & Chmielinski, K. (2018) · arXiv · doi:10.48550/arxiv.1805.03677 · Cited by 175
Neff, G., Tanweer, A., Fiore-Gartland, B., & Osburn, L. (2017) · Big Data · doi:10.1089/big.2016.0050 · Cited by 173
Shankar, S., Halpern, Y., Breck, E. J., Atwood, J. R., Wilson, J., & Sculley, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08536 · Cited by 167
(2019) · doi:10.1145/3351095.3372862 · Cited by 167
The Practitioner's Guide to Data Quality Improvement
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-17212-4 · Cited by 152
Stodden, V., & Miguez, S. (2014) · Journal of Open Research Software · doi:10.5334/jors.ay · Cited by 139
Infrastructure: The Social Value of Shared Resources
Frischmann, B. M. (2012) · OUP Catalogue · Cited by 137
Recoding gender: women's changing participation in computing
(2013) · Choice Reviews Online · doi:10.5860/choice.50-5630 · Cited by 126
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level ConstraintsPreprint
Zhao, J., Wang, T., Yatskar, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09457 · Cited by 122
Winner's Curse? On Pace, Progress, and Empirical Rigor.
Sculley, D., Snoek, J., Wiltschko, A. B., & Rahimi, A. R. O. (2018) · International Conference on Learning Representations · Cited by 117
Cost, benefits and quality of software development documentation: A systematic mapping
Zhi, J., Garousi-Yusifoğlu, V., Sun, B., Garousi, G., Shahnewaz, S., & Ruhe, G. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2014.09.042 · Cited by 112
Provenance Analysis for Missing Answers and Integrity Repairs.
Zaharia, M., Chen, A., Davidson, A., Ghodsi, A., Hong, S. A., Konwinski, A., Murching, S., Nykodym, T., Ogilvie, P., Parkhe, M., et al. (2018) · IEEE Data(base) Engineering Bulletin · Cited by 112
Designing Contestability: Interaction Design, Machine Learning, and Mental Health
Hirsch, T., Merced, K., Narayanan, S., Imel, Z. E., & Atkins, D. C. (2017) · Proceedings of the 2017 Conference on Designing Interactive Systems · doi:10.1145/3064663.3064703 · Cited by 110 · cited 1×
“been motivated as a way of encouraging critical and responsible engagement between various stakeholders, surfacing values that might not otherwise be visible, and safeguarding a system against misuse [68, 103]. However, the ability of a given stakeholder to challenge an automated decision is limited, in large part, by…”
BackgroundWhy we need a philosophy of engineering: a work in progress
Goldman, S. L. (2004) · Interdisciplinary Science Reviews · doi:10.1179/030801804225012572 · Cited by 104
Women also Snowboard: Overcoming Bias in Captioning Models
Burns, K., Hendricks, L. A., Saenko, K., Darrell, T. J., & Rohrbach, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09797 · Cited by 100
Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning
Jo, E. S., & Gebru, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10389 · Cited by 100
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
Denton, R., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020) · arXiv · doi:10.48550/arxiv.2007.07399 · Cited by 100 · cited 3×
“Engineering artefacts are imbued with politics via their roles in sociopolitical systems [57, 147], and so too are ML datasets inseparable from politics [36, 41, 113].”
BackgroundInfluentialSelf-citationDoes Object Recognition Work for Everyone?
DeVries, T., Misra, I., Wang, C., & van der Maaten, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02659 · Cited by 95
Postmortem reviews: purpose and approaches in software engineering
Dingsøyr, T. (2005) · Information and Software Technology · doi:10.1016/j.infsof.2004.08.008 · Cited by 93
Large image datasets: A pyrrhic win for computer vision?
Prabhu, V. U., & Birhane, A. (2020) · arXiv · doi:10.48550/arxiv.2006.16923 · Cited by 91
Engineering problems in machine learning systems
Kuwajima, H., Yasuoka, H., & Nakae, T. (2020) · Machine Learning · doi:10.1007/s10994-020-05872-w · Cited by 86
Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Ashmore, R., Călinescu, R., & Paterson, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04223 · Cited by 83
Livingston, E. (1999) · Social Studies of Science · doi:10.1177/030631299029006003 · Cited by 82
Influence of personality types in software tasks choices
Capretz, L. F., Varona, D., & Raza, A. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.05.050 · Cited by 69
C++ coding standards : 101 rules, guidelines, and best practices
Sutter, H., & Alexandrescu, A. T. (2004) · Cited by 62
The Principle of the Malevolent Hiding Hand; or, the Planning Fallacy Writ Large
Flyvbjerg, B., & Sunstein, C. R. (2016) · Social research · doi:10.1353/sor.2016.0063 · Cited by 61
Theoretical Foundations of Artificial General Intelligence
Wang, P., & Goertzel, B. (2012) · Atlantis thinking machines · doi:10.2991/978-94-91216-62-6 · Cited by 57
From Principles to Practice. An interdisciplinary framework to operationalise AI ethics.
Hallensleben, S., Fetic, L., Fleischer, T., Grünke, Hagendorff, T., Hauer, M. P., Hauschke, A., Heesen, J., Herrmann, M., Hillerbrand, R., et al. (2020) · Cited by 53
Journey to the Moon: The History of the Apollo Guidance Computer
Hall, E. C. (1996) · American Institute of Aeronautics and Astronautics eBooks · doi:10.2514/4.868023 · Cited by 51
SystemDS: A Declarative Machine Learning System for the End-to-End Data Science LifecyclePreprint
Böehm, M., Antonov, I., Baunsgaard, S., Dokter, M., Ginthoer, R. E., Innerebner, K., Klezin, F., Lindstaedt, S., Phani, A., Rath, B., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.02976 · Cited by 49
Proceedings of the Aristotelian Society
Barnes, W. H. F. (1944) · Nature · doi:10.1038/154226b0 · Cited by 47
Philosophy of Engineering: What It Is and Why It Matters
Bulleit, W. M., Schmidt, J. A., Alvi, I. A., Nelson, E. A., & Rodríguez-Nikl, T. (2014) · Journal of Professional Issues in Engineering Education and Practice · doi:10.1061/(asce)ei.1943-5541.0000205 · Cited by 46
Quantifying Gender Bias in Different Corpora
Babaeianjelodar, M., Lorenz, S., Gordon, J., Matthews, J., & Freitag, E. (2020) · Companion Proceedings of the Web Conference 2020 · doi:10.1145/3366424.3383559 · Cited by 43
The algorithm at work? Explanation and repair in the enactment of similarity in art data
Sachs, S. E. (2020) · Information, Communication & Society · doi:10.1080/1369118x.2019.1612933 · Cited by 40
Towards Traceability in Data Ecosystems using a Bill of Materials ModelPreprint
Barclay, I., Preece, A. D., Taylor, I., & Verma, D. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04253 · Cited by 38
Raji, I. D., & Yang, J. (2019) · arXiv · doi:10.48550/arxiv.1912.06166 · Cited by 36
Metrology for AI: From Benchmarks to InstrumentsPreprint
Welty, C., Paritosh, P. K., & Aroyo, L. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01875 · Cited by 36
Information Cultures in the Digital Age
(2016) · Springer eBooks · doi:10.1007/978-3-658-14681-8 · Cited by 32
More Diverse Dialogue Datasets via Diversity-Informed Data Collection
Stasaski, K., Yang, G. H., & Hearst, M. A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.446 · Cited by 32
Mulligan, D. K., Kluttz, D., & Kohli, N. (2019) · doi:10.2139/ssrn.3311894 · Cited by 32 · cited 1×
“ious stakeholders, surfacing values that might not otherwise be visible, and safeguarding a system against misuse [68, 103].”
BackgroundScientific research and big data
Leonelli, S. (2020) · Open Research Exeter (University of Exeter) · Cited by 30
Crossroads of Social Science: The ICPSR 25th Anniversary Volume.
Smith, T., & Eulau, H. H. F. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2072157 · Cited by 28
Curating identifiable data for sharing: The databrary project
Gilmore, R. O., Adolph, K. E., & Millman, D. S. (2016) · New York Scientific Data Summit (NYSDS) · doi:10.1109/nysds.2016.7747817 · Cited by 25
A Survey of Data Quality Measurement and Monitoring Tools
Ehrlinger, L., & Wöß, W. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.850611 · Cited by 25
NLP's Clever Hans Moment has Arrived
Heinzerling, B. (2020) · The Journal of Cognitive Science · doi:10.17791/jcs.2020.21.1.161 · Cited by 19
Deep Reinforcement Learning as Foundation for Artificial General Intelligence
Arel, I. (2012) · Atlantis thinking machines · doi:10.2991/978-94-91216-62-6_6 · Cited by 18
Loshin, D. S. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374225-4.00006-0 · Cited by 18
Infrastructure and the Principle of the Hiding Hand
Anheier, H. K. (2017) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198787310.003.0004 · Cited by 18
Hanusch, F., & Mellado, C. (2014) · International journal of communication · Cited by 17
Pokharel, R. K. (2016) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-1650-7.ch008 · Cited by 17
Why Reliabilism Is not Enough: Epistemic and Moral Justification in Machine Learning
Smart, A., James, L., Hutchinson, B., Wu, S., & Vallor, S. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375866 · Cited by 17
Self-citationContextual Inquiry and Requirements Shaping
Bednár, P., & Welch, C. (2008) · Springer eBooks · doi:10.1007/978-0-387-68772-8_18 · Cited by 15
Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in SummarizationPreprint
Jung, T., Kang, D., Mentch, L., & Hovy, E. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11723 · Cited by 14
Information Systems Development
Mattia, A., & Weistroffer, H. R. (2011) · IGI Global eBooks · doi:10.4018/9781605669847.ch036 · Cited by 13
Engineers Need to Get Real, But Can't: The Role of Models
Alvi, I. A. (2013) · ASCE Structures Congress · doi:10.1061/9780784412848.081 · Cited by 12
The Bias Paradox: Why it's Not Just for Feminists Anymore
Heikes, D. K. (2004) · Synthese · doi:10.1023/b:synt.0000016424.47883.b9 · Cited by 11
Anahideh, H., & Asudeh, A. (2020) · Figshare · doi:10.25417/uic.14869386.v1 · Cited by 11
RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking RecipesPreprint
Yagcioglu, S., Erdem, A., Erdem, E., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00812 · Cited by 10
Fair Active LearningPreprint
Anahideh, H., Asudeh, A., & Thirumuruganathan, S. (2020) · arXiv (Cornell University) · doi:10.25417/uic.14869377 · Cited by 10
Recoding Gender: Women’s Changing Participation in Computing
Hatton, E. (2014) · Contemporary Sociology A Journal of Reviews · doi:10.1177/0094306114545742 · Cited by 9
Accountability Challenges in the Governance of Infrastructure
Jordana, J. (2017) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198787310.003.0003 · Cited by 9
Davis, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.1629 · Cited by 9
Mullenbach, J., Gordon, J. A. R., Peng, N., & May, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1625 · Cited by 8
Ethical and Socially-Aware Data Labels
Beretta, E., Vetrò, A., Lepri, B., & De Martin, J. C. (2019) · Communications in Computer and Information Science · doi:10.1007/978-3-030-11680-4_30 · Cited by 6
Distributed System Design Checklist
Hall, B., & Driscoll, K. R. (2014) · NASA STI Repository (National Aeronautics and Space Administration) · Cited by 5
Editorial 2020 Part II: Data from nowhere?
Lane, S. N. (2019) · Earth Surface Processes and Landforms · doi:10.1002/esp.4775 · Cited by 5
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical ReasoningPreprint
Jian, L., Cui, L., Liu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08124 · Cited by 5
Kanonizing Katz| Commuting and Co-Authoring: How To Be in More Than One Place at the Same Time
Katz, E. (2014) · International journal of communication · Cited by 4
Jacobs, A. Z., & Wallach, H. (2019) · arXiv · doi:10.48550/arxiv.1912.05511
Anahideh, H., Asudeh, A., & Thirumuruganathan, S. (2020) · arXiv · doi:10.48550/arxiv.2001.01796
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · arXiv · doi:10.48550/arxiv.1905.12516
A Survey of Data Quality Measurement and Monitoring Tools
Ehrlinger, L., Rusz, E., & Wöß, W. (2019) · arXiv · doi:10.48550/arxiv.1907.08138
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Zellers, R., Bisk, Y., Schwartz, R., & Choi, Y. (2018) · arXiv · doi:10.48550/arxiv.1808.05326
Research works printed but not linked: 25
- 2. Alain Abran, James W Moore, Pierre Bourque, Robert Dupuis, and L Tripp. 2004. Software engineering body of knowledge. IEEE Computer Society, Angela Burgess (2004).
- 19. Anandhi S Bharadwaj. 2000. A resource-based perspective on information technology capability and firm performance: an empirical investigation. MIS quarterly (2000), 169-196.
- 22. Geoffrey C Bowker and Susan Leigh Star. 2000. Sorting things out: Classification and its consequences. MIT press. · doi:10.5555/380681
- 24. Harvey Brooks. 1994. The relationship between science and technology. Research policy 23, 5 (1994), 477-486.
- 27. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accu- racy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77-91.
- 29. Lawrence Busch. 2014. A dozen ways to get lost in translation: Inherent chal- lenges in large scale data sets. International Journal of Communication 8 (2014), 18.
- 32. Danielle Citron and Frank Pasquale. 2014. The scored society: Due process for automated predictions. Washington Law Review 89 (03 2014), 1-33.
- 34. IEEE Standards Coordinating Committee et al. 1990. IEEE Standard Glossary of Software Engineering Terminology (IEEE Std 610.12-1990). Los Alamitos. CA: IEEE Computer Society 169 (1990).
- 36. Kate Crawford, Mary L Gray, and Kate Miltner. 2014. Critiquing Big Data: Politics, ethics, epistemology (special section introduction). International Journal of Communication 8 (2014), 10.
- 37. Kate Crawford and Trevor Paglen. 2019. Excavating AI: The politics of images in machine learning training sets. Excavating AI (2019).
- 56. Martin Glinz. 2011. A glossary of requirements engineering terminology. Stan- dard Glossary of the Certified Professional for Requirements Engineering (CPRE) Studies and Exam, Version 1 (2011), 56.
- 59. Ben Green. 2020. Data science as political action: Grounding data science in a politics of justice. Available at SSRN 3658431 (2020).
- 83. Nitin Kohli, Renata Barreto, and Joshua A Kroll. 2018. Translation tutorial: a shared lexicon for research and practice in human-centered software systems. In 1st Conference on Fairness, Accountability, and Transparancy. New York, NY, USA, Vol. 7.
- 94. Xiaoxuan Liu, Livia Faes, Aditya U Kale, Siegfried K Wagner, Dun Jack Fu, Alice Bruynseels, Thushika Mahendiran, Gabriella Moraes, Mohith Shamdas, Christoph Kern, et al. 2019. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a system- atic review and meta-analysis. The lancet digital health 1, 6 (2019), e271-e297.
- 106. Helen Nissenbaum. 1996. Accountability in a computerized society. Science and engineering ethics 2, 1 (1996), 25-42.
- 107. Roy Osherove. 2015. The art of unit testing. MITP-Verlags GmbH & Co. KG.
- 119. Rashida Richardson, Jason M Schultz, and Kate Crawford. 2019. Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. NYUL Rev. Online 94 (2019), 15.
- 140. Astra Taylor. 2018. The automation charade. Logic Magazine (2018).
- 143. Joel Walmsley. 2020. Artificial intelligence and the value of transparency. AI & SOCIETY (2020), 1-11.
- 144. Kai Wegrich and Gerhard Hammerschmid. 2017. Infrastructure governance as political choice. The governance of infrastructure (2017), 21-42.
- 10.18653/v1 · doi:10.18653/v1
- 150. Richard Xiao. 2010. Corpus creation. Handbook of Natural Language Processing (2n Revised edition) (2010), 147-165.
- 10.18653/v1 · doi:10.18653/v1
- 153. Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, Fen Xie, and Corey Zumaret. 2018. Accelerating the Machine Learning Lifecycle with MLflow. In IEEE Data Eng. Bull. 41, no. 4. 39-45.
- 157. Matthew Zook, Solon Barocas, Danah Boyd, Kate Crawford, Emily Keller, Seeta Peña Gangadharan, Alyssa Goodman, Rachelle Hollander, Barbara A Koenig, Jacob Metcalf, et al. 2017. Ten simple rules for responsible big data research.
Cited by
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Datasets: A Community Library for Natural Language Processing
Lhoest, Q., Villanova del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., Tunstall, L., et al. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2021.emnlp-demo.21 · Cited by 343
“Datasets can be seen as a form of infrastructure (Hutchinson et al., 2021).”
BackgroundPower to the People? Opportunities and Challenges for Participatory AI
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3551624.3555290 · Cited by 330
Metadata
- Authors
- Hutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M.
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 560-575
- OpenAlex
- W3135371071
- Semantic Scholar
- 300fa071d990d5c48385da39dccea6b7b280edec
- Cited by
- 279
- References
- 123
- Citations by year
- 2026: 25 · 2025: 31 · 2024: 54 · 2023: 73 · 2022: 60 · 2021: 35
- FWCI
- 50.18
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- accountability, software development lifecycle, data work, data documentation, auditing, decision-making, data infrastructure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
