Lessons from archives: strategies for collecting sociocultural data in machine learning
Open accessThe DOI matches its Crossref record; 18 of the 19 references it cites were found in a registry or an index.
Abstract
A growing body of work shows that many problems in fairness, accountability, transparency, and ethics in machine learning systems are rooted in decisions surrounding the data collection and annotation process. In spite of its fundamental nature however, data collection remains an overlooked part of the machine learning (ML) pipeline. In this paper, we argue that a new specialization should be formed within ML that is focused on methodologies for data collection and annotation: efforts that require institutional frameworks and procedures. Specifically for sociocultural data, parallels can be drawn from archives and libraries. Archives are the longest standing communal effort to gather human information and archive scholars have already developed the language and procedures to address and discuss many challenges pertaining to data collection such as consent, power, inclusivity, transparency, and ethics & privacy. We discuss these five key approaches in document collection practices in archives that can inform data collection in sociocultural ML. By showing data collection practices from another field, we encourage ML research to be more cognizant and systematic in data collection and draw from interdisciplinary expertise.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Data provenance frameworks developed in the context of AI and ML have incorporated perspectives from libraries and archives, feminist ethics and even the food industry (Holland et al. 2020;Jo and Gebru 2020;Luccioni et al. 2022).”
From Can AI provenance inform archival provenance? (2026) · cites this paper as Jo and Gebru 2020
The cited source says what the claim attributes to it.
In the citing paper
“Through the work of AI bias scholars who have drawn on frameworks from archives to develop these concepts (e.g., Jo and Gebru 2020), provenance in AI has emerged as a reflective and communicative practice, if not necessarily a foundational one for the broader AI field.”
From Can AI provenance inform archival provenance? (2026) · cites this paper as Jo and Gebru 2020
The cited source says what the claim attributes to it.
In the citing paper
“In dealing with the emerging ethical and legal challenges posed by AI developments, computer scientists have already turned, in one case at least, to archives for an example of ethical conduct in information management.”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [35]
The cited source says what the claim attributes to it.
In the citing paper
“Jo and Gebru [ 35 ] write that archives provide an example of a thoughtful approach to fraught issues of”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [35]
The cited source says what the claim attributes to it.
In the citing paper
“Since training AI on imbalanced datasets data can fuel inequalities”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [154]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Self-citationFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 1×
“For instance, [19] shows trained word embeddings replicating Asian stereotypes in language data from historical Google Books and the Corpus of Historical American English (COHA).”
BackgroundHovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2006) · Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers on XX - NAACL '06 · doi:10.3115/1614049.1614064 · Cited by 806 · cited 1×
“ears as the standard for Part-of-Speech tags [45]. Others include OntoNotes for coreference resolution and named entity recognition (NER) and the EuroParl parallel corpus for machine translation (MT) [30, 40]. Sometimes, researchers and practitioners build on top of existing datasets motivated by availability and conv…”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
The Penn Treebank: annotating predicate argument structure
Marcus, M., Kim, G., Marcinkiewicz, M. A., MacIntyre, R., Bies, A., Ferguson, M., Katz, K., & Schasberger, B. (1994) · Proceedings of the workshop on Human Language Technology - HLT '94 · doi:10.3115/1075812.1075835 · Cited by 742
Community Histories, Community Archives: Some Opportunities and Challenges1
Flinn, A. (2007) · Journal of the Society of Archivists · doi:10.1080/00379810701611936 · Cited by 279
Hamidi, F., Scheuerman, M. K., & Branham, S. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173582 · Cited by 222
Widespread underprovision on Reddit
Gilbert, E. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441866 · Cited by 173
Research works printed but not linked: 26
- ALA. 2008. Code of Ethics of the American Library Association . American Library Association . http://www.ala.org/tools/ethics ALA. 2008. Code of Ethics of the American Library Association. American Library Association. http://www.ala.org/tools/ethics
- ARA. 2018. Code of Ethics . Archives & Records Association (U.K. & Ireland) . https://www.archives.org.uk/membership/code-of-ethics.html ARA. 2018. Code of Ethics. Archives & Records Association (U.K. & Ireland). https://www.archives.org.uk/membership/code-of-ethics.html
- Richard Breitman and Norman J.W . Goda . 2010 . Hitler's Shadow: Nazi War Criminals, U.S. Intelligence, and the Cold War. National Archives . Richard Breitman and Norman J.W. Goda. 2010. Hitler's Shadow: Nazi War Criminals, U.S. Intelligence, and the Cold War. National Archives.
- Caroline Brown . 2014. Archives and Recordkeeping: Theory into Practices . Facet Publishing . Caroline Brown. 2014. Archives and Recordkeeping: Theory into Practices. Facet Publishing.
- Joy Buolamwini and Timnit Gebru . 2018 . Gender shades: Intersectional accuracy disparities in commercial gender classification . In Conference on Fairness, Accountability and Transparency. 77--91 . Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. 77--91.
- Maygene F. Daniels and Timothy Walch . 1984 . A Modern Archives Reader: Basic Readings on Archival Theory and Practice. National Archives and Records Service . Maygene F. Daniels and Timothy Walch. 1984. A Modern Archives Reader: Basic Readings on Archival Theory and Practice. National Archives and Records Service.
- Nancy Freeman and Holly Geist . 2014. FOIA Request . Case Studies in Archival Ethics ( 2014 ). Nancy Freeman and Holly Geist. 2014. FOIA Request. Case Studies in Archival Ethics (2014).
- Clare Garvie , Bedoya Alvaro , and Jonathan Frankle . 2016. The perpetual line-up: Unregulated police face recognition in America. Georgetown Law , Center on Privacy & Technology. Clare Garvie, Bedoya Alvaro, and Jonathan Frankle. 2016. The perpetual line-up: Unregulated police face recognition in America. Georgetown Law, Center on Privacy & Technology.
- Timnit Gebru , Jamie Morgenstern , Briana Vecchione , Jennifer Wortman Vaughan , Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018 . Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
- Tracy B. Grimm and Chon A. Noriega . 2013 . Documenting Regional Latino Arts and Culture: Case Studies for a Collaborative , Community-Oriented Approach. The American Archivist 76 (2013), 95 -- 112 . Tracy B. Grimm and Chon A. Noriega. 2013. Documenting Regional Latino Arts and Culture: Case Studies for a Collaborative, Community-Oriented Approach. The American Archivist 76 (2013), 95--112.
- Ailsa C. Holland and Elizabeth Mullins . 2013 . Archives and Archivists 2: Current Trends, New Voices. Four Courts Press . Ailsa C. Holland and Elizabeth Mullins. 2013. Archives and Archivists 2: Current Trends, New Voices. Four Courts Press.
- Valerie Horton and Greg Pronevitz . 2015 . Library Consortia: Models for Collaboration and Sustainability. ALA Editions. Valerie Horton and Greg Pronevitz. 2015. Library Consortia: Models for Collaboration and Sustainability. ALA Editions.
- ICRM. 2016. Code of Ethics . Institute of Certified Records Managers. https ://www.icrm.org/code-of-ethics ICRM. 2016. Code of Ethics. Institute of Certified Records Managers. https://www.icrm.org/code-of-ethics
- Os Keyes , Nikki Stevens , and Jacqueline Wernimont . 2019. The Government Is Using the Most Vulnerable People to Test Facial Recognition Software. Slate Magazine ( 2019 ). Os Keyes, Nikki Stevens, and Jacqueline Wernimont. 2019. The Government Is Using the Most Vulnerable People to Test Facial Recognition Software. Slate Magazine (2019).
- Christopher Kitching and Ian Hart . 1995. Collection Policy Statements. Journal of the Society of Archivists 16 ( 1995 ). http://www.nationalarchives.gov.uk/documents/archives/archive-collection-policy.pdf Christopher Kitching and Ian Hart. 1995. Collection Policy Statements. Journal of the Society of Archivists 16 (1995). http://www.nationalarchives.gov.uk/documents/archives/archive-collection-policy.pdf
- Philipp Koehn . 2005 . Europarl: A parallel corpus for statistical machine translation. In MT summit , Vol. 5 . 79--86. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In MT summit, Vol. 5. 79--86.
- Katherine McCardwell . 2014. Intellectual Propoerty Concerns in Undocumented Corporate Collections. Case Studies in Archival Ethics ( 2014 ). Katherine McCardwell. 2014. Intellectual Propoerty Concerns in Undocumented Corporate Collections. Case Studies in Archival Ethics (2014).
- Paul Mozur . 2019. One Month , 500,000 Face Scans : How China Is Using A.I. to Profile a Minority. The New York Times ( 2019 ). Paul Mozur. 2019. One Month, 500,000 Face Scans: How China Is Using A.I. to Profile a Minority. The New York Times (2019).
- MS. 1999. The Manuscript Society of Code of Ethics Adopted by the Board of Trustees May 26, 1999 . The Manuscript Society . manuscript.org/about/code-of-ethics MS. 1999. The Manuscript Society of Code of Ethics Adopted by the Board of Trustees May 26, 1999. The Manuscript Society. manuscript.org/about/code-of-ethics
- Timothy D. Pyatt . 2015. The Harding Affair Letters: How One Archivist Took Every Measure Possible To Ensure Their Preservation. Case Studies in Archival Ethics ( 2015 ). Timothy D. Pyatt. 2015. The Harding Affair Letters: How One Archivist Took Every Measure Possible To Ensure Their Preservation. Case Studies in Archival Ethics (2015).
- Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 ( 2019 ). Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019).
- Ellen M. Ryan . 2014. Identifying Culturally Sensitive American Indian Material in a Non-tribal Institution. Case Studies in Archival Ethics ( 2014 ). Ellen M. Ryan. 2014. Identifying Culturally Sensitive American Indian Material in a Non-tribal Institution. Case Studies in Archival Ethics (2014).
- SAA. 2011. SAA Core Values Statement and Code of Ethics . Society of American Archivists . https://www2.archivists.org/statements/saa-core-values-statement-and-code-of-ethics SAA. 2011. SAA Core Values Statement and Code of Ethics. Society of American Archivists. https://www2.archivists.org/statements/saa-core-values-statement-and-code-of-ethics
- SAA. 2013 . Describing Archives: A Content Standard ( second ed.). Society of American Archivists . SAA. 2013. Describing Archives: A Content Standard (second ed.). Society of American Archivists.
- Harini Suresh and John V . Guttag . 2019 . A Framework for Understanding Unintended Consequences of Machine Learning. ArXiv abs/1901.10002 (2019). Harini Suresh and John V. Guttag. 2019. A Framework for Understanding Unintended Consequences of Machine Learning. ArXiv abs/1901.10002 (2019). · arXiv
- Aditya Tripathi and Jawahar Lal . 2016 . Library Consortia: Practical Guide for Library Managers . Chandos Publishing . Aditya Tripathi and Jawahar Lal. 2016. Library Consortia: Practical Guide for Library Managers. Chandos Publishing.
Cited by
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Self-citationOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Directions in abusive language training data, a systematic review: Garbage in, garbage out
Vidgen, B., & Derczynski, L. (2020) · PLOS ONE · doi:10.1371/journal.pone.0243300 · Cited by 318
Power to the People? Opportunities and Challenges for Participatory AI
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3551624.3555290 · Cited by 330
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306
“Jo and Gebru (2020) discuss parallels between creating historical archives and the curation of machine learning datasets including pretraining corpora.”
BackgroundAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Hutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
Technology readiness levels for machine learning systems
Lavin, A., Gilligan-Lee, C. M., Visnjic, A., Ganju, S., Newman, D., Ganguly, S., Lange, D., Baydin, A. G., Sharma, A., Gibson, A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33128-9 · Cited by 209
Can HR adapt to the paradoxes of artificial intelligence?
Charlwood, A., & Guenole, N. (2022) · Human Resource Management Journal · doi:10.1111/1748-8583.12433 · Cited by 271
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
On the genealogy of machine learning datasets: A critical history of ImageNet
Denton, E., Hanna, A., Amironesei, R., Smart, A., & Nicole, H. (2021) · Big Data & Society · doi:10.1177/20539517211035955 · Cited by 221
“Data work is heavily under-incentivized, with most attention being paid to algorithmic developments (Jo and Gebru, 2020; Hutchinson et al., 2021; Sambasivan et al., 2021) and publications that focus solely on dataset creation tend to be devalued within traditional peer-review processes…”
BackgroundDo Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development
Scheuerman, M. K., Hanna, A., & Denton, R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476058 · Cited by 216
A sociotechnical view of algorithmic fairness
Dolata, M., Feuerriegel, S., & Schwabe, G. (2022) · Information Systems Journal · doi:10.1111/isj.12370 · Cited by 160
Simulation Intelligence: Towards a New Generation of Scientific MethodsPreprint
Lavin, A., Krakauer, D. C., Zenil, H., Gottschlich, J., Mattson, T., Brehmer, J., Anandkumar, A., Choudry, S., Rocki, K., Baydin, A. G., et al. (2021) · CaltechAUTHORS (California Institute of Technology) · doi:10.48550/arxiv.2112.03235 · Cited by 221
Davis, J. L., Williams, A., & Yang, M. W. (2021) · Big Data & Society · doi:10.1177/20539517211044808 · Cited by 143
Archives and AI: An Overview of Current Debates and Future Perspectives
Colavizza, G., Blanke, T., Jeurgens, C., & Noordegraaf, J. (2022) · Journal on Computing and Cultural Heritage · doi:10.1145/3479010 · Cited by 162
Algorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 224
Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157
“Applying data pre-processing techniques [68, 69] to improve the quality of data (regarding diversity and inclusion) often proves more effective than dealing with unfairness resulting from feeding biased data into models for decision making and predictions [54, 55].”
BackgroundMethodInfluentialPolicy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 160
Synthetic data as an enabler for machine learning applications in medicine
Rajotte, J.-F., Bergen, R., Buckeridge, D. L., El Emam, K., Ng, R., & Strome, E. (2022) · iScience · doi:10.1016/j.isci.2022.105331 · Cited by 124
Klein, L. F., & D’Ignazio, C. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658543 · Cited by 173
Retiring Adult: New Datasets for Fair Machine LearningPreprint
Ding, F., Hardt, M., Miller, J. A., & Schmidt, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04884 · Cited by 599
Challenges in Automated Debiasing for Toxic Language Detection
Zhou, X., Sap, M., Swayamdipta, S., Choi, Y., & Smith, N. A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.274 · Cited by 202
AI and the Everything in the Whole Wide World Benchmark
Raji, I. D., Bender, E. M., Paullada, A., Denton, E., & Hanna, A. (2021) · arXiv · doi:10.48550/arxiv.2111.15366 · Cited by 127
“More generally, dataset development practices within machine learning have been characterized by a “laissez-faire” attitude, rather than a more thoughtful, intentional, and curatorial approach [Jo and Gebru, 2020].”
BackgroundMethodRethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
Weinberg, L. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13196 · Cited by 127
Wang, A., Ramaswamy, V., & Russakovsky, O. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533101 · Cited by 171
Wusu, O. (2024) · AI and Ethics · doi:10.1007/s43681-024-00432-7 · Cited by 115
Reframing data ethics in research methods education: a pathway to critical data literacy
Atenas, J., Havemann, L., & Timmermann, C. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00380-y · Cited by 111
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
Denton, R., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020) · arXiv · doi:10.48550/arxiv.2007.07399 · Cited by 100
Challenges and Strategies in Cross-Cultural NLP
Hershcovich, D., Frank, S., Lent, H., de Lhoneux, M., Abdou, M., Brandl, S., Bugliarello, E., Cabello Piqueras, L., Chalkidis, I., Cui, R., et al. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.482 · Cited by 99
The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations
Balagopalan, A., Zhang, H., Hamidieh, K., Hartvigsen, T., Rudzicz, F., & Ghassemi, M. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533179 · Cited by 133
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 77
Understanding and Evaluating Racial Biases in Image Captioning
Zhao, D., Wang, A., & Russakovsky, O. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01456 · Cited by 169
Unlocking digital archives: cross-disciplinary perspectives on AI and born-digital data
Jaillant, L., & Caputo, A. (2022) · AI & Society · doi:10.1007/s00146-021-01367-x · Cited by 96
Razi, A., Kim, S., Alsoubai, A., Stringhini, G., Solorio, T., De Choudhury, M., & Wisniewski, P. J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479609 · Cited by 95
A Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms
Kim, S., Razi, A., Stringhini, G., Wisniewski, P. J., & De Choudhury, M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476066 · Cited by 79
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
Gururangan, S., Card, D., Dreier, S. K., Gade, E. K., Wang, L. Z., Wang, Z., Zettlemoyer, L., & Smith, N. A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.165 · Cited by 133
A Survey of Data Quality Requirements That Matter in ML Development Pipelines
Priestley, M., O’donnell, F., & Simperl, E. (2023) · Journal of Data and Information Quality · doi:10.1145/3592616 · Cited by 86
The Role of Explainable AI in the Research Field of AI Ethics
Vainio-Pekka, H., Agbese, M., Jantunen, M., Vakkuri, V., Mikkonen, T., Rousi, R. A., & Abrahamsson, P. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3599974 · Cited by 99
Responsible AI in Africa: Challenges and Opportunities
(2023) · Social and Cultural Studies of Robots and AI · doi:10.1007/978-3-031-08215-3 · Cited by 82
Thylstrup, N. B. (2022) · Media Culture & Society · doi:10.1177/01634437211060226 · Cited by 97
Responsible AI in Africa—Challenges and Opportunities
Okolo, C. T., Aruleba, K., & Obaido, G. (2023) · Social and Cultural Studies of Robots and AI · doi:10.1007/978-3-031-08215-3_3 · Cited by 79
Shen, H., Deng, W. H., Chattopadhyay, A., Wu, Z. S., Wang, X., & Zhu, H. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445971 · Cited by 102
Robots Enact Malignant Stereotypes
Hundt, A., Agnew, W. S., Zeng, V., Kacianka, S., & Gombolay, M. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533138 · Cited by 104
Boyd, K., & Andalibi, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579528 · Cited by 88
Foundation models are platform models: Prompting and the political economy of AI
Burkhardt, S., & Rieder, B. (2024) · Big Data & Society · doi:10.1177/20539517241247839 · Cited by 89
IoT Health Devices: Exploring Security Risks in the Connected Landscape
Affia, A.-A. O., Finch, H., Jung, W., Samori, I. A., Potter, L. N., & Palmer, X. (2023) · IoT · doi:10.3390/iot4020009 · Cited by 88
Co-creating digital art with generative AI in K-9 education: Socio-material insights
Vartiainen, H., Tedre, M., & Jormanainen, I. (2023) · International Journal of Education Through Art · doi:10.1386/eta_00143_1 · Cited by 73
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Narayanan, A., & Russakovsky, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_43 · Cited by 98
Whose AI Dream? In search of the aspiration in data annotation.
Wang, D., Prabhat, S., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502121 · Cited by 70
“…and responsible AI, data documentation such as datasheet [25] and data nutrition label [30], efforts to document the sociological aspects of data [32] and documentation to promote reflexivity [39] still put the emphasis on data, and its impact on model design and development, rarely does the…”
BackgroundWhat’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A., & Viviano, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.18653/v1/2021.acl-short.24 · Cited by 69
“…likely require the community to adopt more mindful data collection practices (Gehman et al., 2020; Bender and Friedman, 2018; Gebru et al., 2018; Jo and Gebru, 2020; Paullada et al., 2020; Bender et al., 2021), establish standardized filtering pipelines for corpora (Roziewski and Stokowiec,…”
BackgroundCan Workers Meaningfully Consent to Workplace Wellbeing Technologies?
Chowdhary, S., Kawakami, A., Gray, M. L., Suh, J., Olteanu, A., & Saha, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594023 · Cited by 84
“They only care to show us the wheelchair”: disability representation in text-to-image AI models
Mack, K. A., Qadri, R., Denton, R., Kane, S. K., & Bennett, C. L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642166 · Cited by 67
Smits, T., & Wevers, M. (2023) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqad008 · Cited by 83
Lifting the curtain: Strategic visibility of human labour in AI-as-a-Service
Newlands, G. (2021) · Big Data & Society · doi:10.1177/20539517211016026 · Cited by 66
Paulus, D., Fathi, R., Fiedrich, F., de Walle, B. V., & Comes, T. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10241-0 · Cited by 74
Epistemic injustice and data science technologies
Symons, J., & Alvarado, R. (2022) · Synthese · doi:10.1007/s11229-022-03631-z · Cited by 65
Mittal, S., Thakral, K., Singh, R., Vatsa, M., Glaser, T., Ferrer, C. C., & Hassner, T. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00874-y · Cited by 79
Leavy, S., Σιαπέρα, Ε., & O’Sullivan, B. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462598 · Cited by 81
Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias
Wyllie, S., Shumailov, I., & Papernot, N. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659029 · Cited by 114
Perrotta, C., Selwyn, N., & Ewin, C. (2024) · New Media & Society · doi:10.1177/14614448221075296 · Cited by 59
Fair Generative Modeling via Weak SupervisionPreprint
Choi, K., Grover, A., Singh, T., Shu, R., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12008 · Cited by 124
Generative Ai in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity
Novelli, C., Casolari, F., Hacker, P., Spedicato, G., & Floridi, L. (2024) · doi:10.2139/ssrn.4821952 · Cited by 57
Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review
Li, M., Enkhtur, A., Yamamoto, B. A., Cheng, F., & Chen, L. (2025) · Open Praxis · doi:10.55982/openpraxis.17.1.750 · Cited by 65
The Limits of Global Inclusion in AI DevelopmentPreprint
Chan, A., Okolo, C. T., Terner, Z., & Wang, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01265 · Cited by 75
Softcite dataset: A dataset of software mentions in biomedical and economic research publications
Du, C., Cohoon, J., Lopez, P., & Howison, J. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24454 · Cited by 56
Toward accountable human-centered AI: rationale and promising directions
Qadir, J., Islam, M. Q., & Al-Fuqaha, A. (2022) · Journal of Information, Communication and Ethics in Society · doi:10.1108/jices-06-2021-0059 · Cited by 46
Datasheets for Digital Cultural Heritage Datasets
Alkemade, H., Claeyssens, S., Colavizza, G., Freire, N., Lehmann, J., Neudecker, C., Osti, G., & van Strien, D. (2023) · Journal of Open Humanities Data · doi:10.5334/johd.124 · Cited by 69
Participation versus scale: Tensions in the practical demands on participatory AI
Young, M., Ehsan, U., Singh, R., Tafesse, E., Gilman, M. E., Harrington, C., & Metcalf, J. (2024) · First Monday · doi:10.5210/fm.v29i4.13642 · Cited by 77
Changing the World by Changing the Data
Rogers, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.170 · Cited by 51
“In this context, “data curation” means selecting data based on its sociocultural characteristics (Jo and Gebru, 2020).”
BackgroundA hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
“…documentation artefacts ( e.g., Datasheets for Datasets [30], Data Cards [77], Data Statements [5]) and archival artefacts of the decisions made [48] during the annotation process were proposed as a mechanism to foster deliberative accountability [74], to make explicit the tacit knowledge in…”
BackgroundHandling and Presenting Harmful Text in NLP Research
Kirk, H. R., Birhane, A., Vidgen, B., & Derczynski, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.35 · Cited by 88
Hullman, J. R., Kapoor, S., Nanayakkara, P., Gelman, A., & Narayanan, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534196 · Cited by 59
Causal Feature Selection for Algorithmic Fairness
Galhotra, S., Shanmugam, K., Sattigeri, P., & Varshney, K. R. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517909 · Cited by 65
Hagendorff, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09573-8 · Cited by 47
FACET: Fairness in Computer Vision Evaluation Benchmark
Gustafson, L., Rolland, C., Ravi, N., Duval, Q., Adcock, A., Fu, C.-Y., Hall, M. A., & Ross, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01863 · Cited by 95
Overwriting Pretrained Bias with Finetuning Data
Wang, A., & Russakovsky, O. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00366 · Cited by 77
Generative AI in EU Law: Liability, Privacy, Intellectual Property, and CybersecurityPreprint
Novelli, C., Casolari, F., Hacker, P., Spedicato, G. A., & Floridi, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.07348 · Cited by 56
When Fair Ranking Meets Uncertain Inference
Ghosh, A., Dutt, R., & Wilson, C. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462850 · Cited by 58
The Craft and Coordination of Data Curation: Complicating Workflow Views of Data Science
Thomer, A. K., Akmon, D., York, J., Tyler, A. R. B., Polasek, F., Lafia, S., Hemphill, L., & Yakel, E. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555139 · Cited by 44
Social Inclusion in Curated Contexts: Insights from Museum Practices
Huang, H.-Y., & Liem, C. C. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533095 · Cited by 56
AI for social good and the corporate capture of global development
Iazzolino, G., & Stremlau, N. (2024) · Information Technology for Development · doi:10.1080/02681102.2023.2299351 · Cited by 47
2026 Top Trends in Academic Libraries: A Review of the Trends and Issues
2021–22 ACRL Research Planning and Review Committee (2022) · College & Research Libraries News · doi:10.5860/crln.83.6.243 · Cited by 41
Provocations from the Humanities for Generative AI ResearchPreprint
Klein, L., Martin, M. J., Brock, A., Antoniak, M., Walsh, M., Johnson, J. M., Tilton, L., & Mimno, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.19190 · Cited by 45
Measuring Representational Harms in Image Captioning
Wang, A., Barocas, S., Laird, K., & Wallach, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533099 · Cited by 40
Collecting a Large Scale Dataset for Classifying Fake News Tweets Using Weak Supervision
Helmstetter, S., & Paulheim, H. (2021) · Future Internet · doi:10.3390/fi13050114 · Cited by 43
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38
Computer Vision and Conflicting Values: Describing People with Automated Alt Text
Hanley, M., Barocas, S., Levy, K., Azenkot, S., & Nissenbaum, H. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462620 · Cited by 38
Gender Artifacts in Visual Datasets
Meister, N., Zhao, D., Wang, A., Ramaswamy, V., Fong, R., & Russakovsky, O. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00446 · Cited by 55
How Text-to-Image Generative AI Is Transforming Mediated Action
Vartiainen, H., & Tedre, M. (2024) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2024.3355808 · Cited by 42
(2022) · doi:10.1145/3491102.3501868 · Cited by 36
“These broader questions around data gathering have long been the focus of the field of critical data studies, which has closely examined how the context around data—the people, institutions, instruments, policies, finances, and more—impact data collected and their use [20, 44, 48, 50].”
BackgroundThiel, S., Bernhardt, J., & Medien, B. D. B. F. K. U. (2023) · Edition Museum · doi:10.1515/9783839467107 · Cited by 36
Kotsis, K. Τ. (2024) · European Journal of Contemporary Education and E-Learning · doi:10.59324/ejceel.2024.2(3).04 · Cited by 41
Towards intellectual freedom in an AI Ethics Global Community
Ebell, C., Baeza-Yates, R., Benjamins, R., Cai, H., Coeckelbergh, M., Duarte, T., Hickok, M., Jacquet, A., Kim, A., Krijger, J., et al. (2021) · AI and Ethics · doi:10.1007/s43681-021-00052-5 · Cited by 34
ChatGPT and imaginaries of the future of education: insights of Finnish teacher educators
Vartiainen, H., Valtonen, T., Kahila, J., & Tedre, M. (2024) · Information and Learning Sciences · doi:10.1108/ils-10-2023-0146 · Cited by 34
Metadata
- Authors
- Jo, E. S., & Gebru, T.
- Affiliations
- Stanford University (US)
- Google (United States) (US)
- ORCID
- Gebru, Timnit
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 306-316
- OpenAlex
- W4288083800
- Semantic Scholar
- 3dea5fbc7b4b304d57881f8d50c1c0438aef6996
- Cited by
- 240
- References
- 19
- Citations by year
- 2026: 26 · 2025: 37 · 2024: 43 · 2023: 49 · 2022: 44 · 2021: 37 · 2020: 4
- FWCI
- 34.75
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Data Analysis and Archiving
- Social Sciences › Arts and Humanities › Conservation › Digital and Traditional Archives Management
- Keywords
- data collection, data annotation, archives, libraries, machine learning ethics, consent, inclusivity, transparency, privacy, fairness and accountability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.