Ethical Data Curation for AI: An Approach based on Feminist Epistemology and Critical Theories of Race
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
The potential for bias embedded in data to lead to the perpetuation of social injustice though Artificial Intelligence (AI) necessitates an urgent reform of data curation practices for AI systems, especially those based on machine learning. Without appropriate ethical and regulatory frameworks there is a risk that decades of advances in human rights and civil liberties may be undermined. This paper proposes an approach to data curation for AI, grounded in feminist epistemology and informed by critical theories of race and feminist principles. The objective of this approach is to support critical evaluation of the social dynamics of power embedded in data for AI systems. We propose a set of fundamental guiding principles for ethical data curation that address the social construction of knowledge, call for inclusion of subjugated and new forms of knowledge, support critical evaluation of theoretical concepts within data and recognise the reflexive nature of knowledge. In developing this ethical framework for data curation, we aim to contribute to a virtue ethics for AI and ensure protection of fundamental and human rights.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“[52] who uncovered racial bias in an AI system used to support healthcare decisions due to the training data captured historical patterns of access to healthcare”
MethodThe Essential Tension: Selected Studies in Scientific Tradition and Change
Kuhn, T. S. (1977) · doi:10.7208/chicago/9780226217239.001.0001 · Cited by 3,861
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 1×
“Approaches to ensuring fairness in AI have broadly taken a normative approach through the development of ethical guidelines and principles [27, 73].”
BackgroundCertifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“[57] the issues are in large part due to an abstraction of the concept of fairness away from the social context within which they are to be deployed.”
BackgroundRoutledge International Handbook of Participatory Design
(2012) · doi:10.4324/9780203108543 · Cited by 1,606
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305 · cited 1×
“Methods that address bias in data include methods for data augmentation, re-sampling, re-weighting, swapping labels and removing dependencies between characteristics with the aim of achieving neutrality [19, 34].”
MethodWord embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 1×
“covering 100 years dating from 1910 [22].”
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 1×
“[36] considered three different fairness conditions that capture what it means for an algorithm to be fair and avoid exhibiting bias.”
BackgroundInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Technology and the Virtues: A Philosophical Guide to a Future Worth Wanting
Vallor, S. (2016) · doi:10.1093/acprof:oso/9780190498511.001.0001 · Cited by 910
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
(2024) · doi:10.1145/3613904 · Cited by 896
Lambrecht, A., & Tucker, C. (2019) · Management Science · doi:10.1287/mnsc.2018.3093 · Cited by 854 · cited 1×
“of AI to mirror and even exacerbate the discriminatory behaviour that civil rights movements have fought against [15, 38, 51].”
BackgroundDetecting depression and mental illness on social media: an integrative review
Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.005 · Cited by 722 · cited 1×
“[26, 65]) could, if applied to the whole population, have the effect”
BackgroundRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Gender Trouble: Feminism and the Subversion of Identity
Butler, J. (2011) · doi:10.4324/9780203824979 · Cited by 613
Lum, K., & Isaac, W. (2016) · Significance · doi:10.1111/j.1740-9713.2016.00960.x · Cited by 611
Design Justice: towards an intersectional feminist framework for design theory and practice
Costanza-Chock, S. (2018) · Proceedings of DRS · doi:10.21606/drs.2018.679 · Cited by 325
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240 · cited 1×
“The framework we set out may be employed within the context of the data curation process set out by Jo and Gebru [33], who proposed a new specialisation in data curation for AI based on archival methods.”
MethodHow computer systems embody values
Nissenbaum, H. (2001) · Computer · doi:10.1109/2.910905 · Cited by 232
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · cited 2×
“This will necessitate a participatory or co-design approach, which draws upon Constanza-Chock’s [12] approach but also builds upon other work, for example Katell et al. [35].”
BackgroundMethodGebru, T. (2020) · The Oxford Handbook of Ethics of AI · doi:10.1093/oxfordhb/9780190067397.013.16 · Cited by 111
Cited by
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Alexander, C. S., Yarborough, M., & Smith, A. (2024) · Precision Agriculture · doi:10.1007/s11119-023-10063-3 · Cited by 84
Brewer, R. N. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519642 · Cited by 43
Metadata
- Authors
- Leavy, S., Siapera, E., & O'Sullivan, B.
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 695-703
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3192254170
- Cited by
- 50
- References
- 24
- FWCI
- 4.77
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.