Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Open source software communities are a significant site of AI development, but “Ethical AI” discourses largely focus on the problems that arise in software produced by private companies. Design, policy and tooling interventions to encourage “Ethical AI” based on studies in private companies risk being ill-suited for an open source context, which operates under radically different organizational structures, cultural norms, and incentives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 3×
“Systems may adhere to such principles while still being patently unethical [42], and convergence on principles risks obscuring political and normative disagreements [55], or focuses “Ethical AI” scrutiny on AI design rather than the business uses it enables [30].”
BackgroundCodebook Development for Team-Based Qualitative Analysis
MacQueen, K. M., McLellan, E., Kay, K., & Milstein, B. (1998) · CAM Journal · doi:10.1177/1525822x980100020301 · Cited by 1,675 · cited 1×
“After data collection, all interviews were transcribed, and then the first author examined possible relationships between themes in this analytic memo, iteratively going back to the data to test out these possible structures, before settling on an inductive hierarchical coding frame [48, 54, 76].”
MethodSocial coding in GitHub: transparency and collaboration in an open software repository
Dabbish, L., Stuart, C., Tsay, J., & Herbsleb, J. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145396 · Cited by 999
Self-citationImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“Studies to help AI practitioners improve fairness [22, 34], such as checklists to solve organizational challenges [49], are often based on the needs of AI practitioners in private companies, but some studies also focus on the needs of public sector [80] or academic institutions [61].”
BackgroundTwo Bits: The Cultural Significance of Free Software
(2008) · Choice Reviews Online · doi:10.5860/choice.46-2158 · Cited by 709 · cited 3×
“This centers the developer’s freedom to choose how the product of their labor is used, away from the user’s freedom to use the software for “any purpose”, with the goal of using licenses to foster that “make it easy for the user to do the right thing” [41].”
BackgroundVeale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1×
“Studies to help AI practitioners improve fairness [22, 34], such as checklists to solve organizational challenges [49], are often based on the needs of AI practitioners in private companies, but some studies also focus on the needs of public sector [80] or academic institutions [61].”
BackgroundSupply Chain Specific? Understanding the Patchy Success of Ethical Sourcing Initiatives
Roberts, S. (2003) · Journal of Business Ethics · doi:10.1023/a:1023395631811 · Cited by 472 · cited 1×
“Supply chains can help reason about upstream [65] and downstream harms in [26] in offline contexts, and the UN has published actions companies should take to mitigate human rights violations in supply chains [67].”
BackgroundMadaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · cited 1×
“Studies to help AI practitioners improve fairness [22, 34], such as checklists to solve organizational challenges [49], are often based on the needs of AI practitioners in private companies, but some studies also focus on the needs of public sector [80] or academic institutions [61].”
BackgroundToward an Ethics of Algorithms: Convening, Observation, Probability, and Timeliness
Ananny, M. (2016) · Science, Technology, & Human Values · doi:10.1177/0162243915606523 · Cited by 395
Research works printed but not linked: 62
- 2010. The 82nd Academy Awards (2010) Nominees and Winners. http://www.oscars.org/awards/academyawards/legacy/ceremony/82nd-winners.html/. The Academy of Motion Picture Arts and Sciences (2010). Accessed: 2022-004-25.
- 2019. Google Will Not Renew Pentagon Contract That Upset Employees. https://deon.drivendata.org. DrivenData (2019). Accessed: 2021-07-7.
- Henry Ajder, Giorgio Patrini, Francesco Cavalli, and Laurence Cullen. 2019. The state of deepfakes: Landscape, threats, and impact. Amsterdam: Deeptrace(2019).
- 10.1109/MSP.2015.72 · DOI
- Shaosong Ou Alexander Hars. 2002. Working for free? Motivations for participating in open-source projects. International journal of electronic commerce 6, 3 (2002), 25–39.
- 10.1145/3306618.3314243 · DOI · Paper page
- Mike Ananny and Kate Crawford. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. new media & society 20, 3 (2018), 973–989.
- Hannah Arendt. 1987. Collective responsibility. In Amor mundi. Springer, 43–50.
- Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-François Bonnefon, and Iyad Rahwan. 2018. The moral machine experiment. Nature 563, 7729 (2018), 59–64.
- Justin B Biddle. 2020. On predicting recidivism: epistemic risk tradeoffs and values in machine learning. Canadian Journal of Philosophy(2020) 1–21.
- Nick Bostrom. 2017. Strategic implications of openness in AI development. Global policy 8, 2 (2017), 135–148.
- Marcus Burkhardt. 2019. Mapping the Democratization of AI on GitHub. A First Approach. (2019). · doi:10.1515/9783839447192-013
- 10.1145/3376898 · DOI · Paper page
- E Gabriella Coleman. 2012. Coding freedom: The ethics and aesthetics of hacking. Princeton University Press.
- 10.1145/3290607.3299057 · DOI
- Henriette Cramer, Jenn Wortman Vaughan, Ken Holstein, Hanna Wallach, Jean Garcia-Gathright, Hal Daumé III, Miroslav Dudík, and Sravana Reddy. 2019. Challenges of incorporating algorithmic fairness into industry practice. FAT* Tutorial (2019).
- 10.17705/1thci.00133 · DOI
- Jean Macchiaroli Eggen and John G Culhane. 2002. Gun Torts: Defining a cause of action for victims in suits against gun manufacturers. NCL Rev. 81(2002), 115.
- 10.2307/2024543 · DOI · Paper page
- R Stuart Geiger. 2017. Summary analysis of the 2017 github open source survey. arXiv preprint arXiv:1706.02777(2017). · DOI
- Frances S Grodzinsky, Keith Miller, and Marty J Wolf. 2003. Ethical issues in open source software. Journal of Information, Communication and Ethics in Society (2003).
- Guido Hertel, Sven Niedner, and Stefanie Herrmann. 2003. Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel. Research policy 32, 7 (2003), 1159–1177.
- Bill Hibbard. 2008. Open source AI. Frontiers in Artificial Intelligence and Applications 171 (2008), 473.
- 10.1016/j.dss.2012.02.005 · DOI
- Don Ihde. 1990. Technology and the lifeworld: From garden to earth. (1990). · DOI · Paper page
- 10.1177/0049124114523396 · DOI · Paper page
- Jiji. 2020. The RIAA is coming for the YouTube downloaders. https://www.japantimes.co.jp/news/2020/10/02/national/crime-legal/two-men-arrested-deepfake-pornography-videos/. The Japan Times (2 Oct. 2020). Accessed: 2021-07-1.
- 10.1145/3290607.3310433 · DOI · Paper page
- Jan Kietzmann, Linda W Lee, Ian P McCarthy, and Tim C Kietzmann. 2020. Deepfakes: Trick or treat?Business Horizons 63, 2 (2020), 135–146.
- Bruno Latour Wiebe E Bijker and John Law. 1992. Shaping technology/building society: Studies in sociotechnical change. W. Bijker & J. Law (Eds.)(1992) 225–258.
- Yvonna S Lincoln, Susan A Lynham, and Egon G Guba. 2011. Paradigmatic controversies, contradictions, and emerging confluences, revisited. The Sage handbook of qualitative research 4 (2011), 97–128.
- Zachary C Lipton. 2018. The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.Queue 16, 3 (2018), 31–57.
- 10.1145/2441776.2441794 · DOI
- 10.1353/sor.2019.0022 · DOI · Paper page
- Lewis Mumford. 1970. The pentagon of power. Vol. 2. Houghton Mifflin Harcourt P.
- Robin Murphy and David D Woods. 2009. Beyond Asimov: the three laws of responsible robotics. IEEE intelligent systems 24, 4 (2009), 14–20.
- Helen Nissenbaum. 2017. Computing and accountability. In Computer Ethics. Routledge, 273–280.
- Wonseok Oh and Sangyong Jeon. 2007. Membership herding and network stability in the open source community: The Ising perspective. Management science 53, 7 (2007), 1086–1101.
- Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 23–43.
- 10.1080/1369118X.2020.1713842 · DOI · Paper page
- Arnold Pacey. 1983. The culture of technology. MIT press.
- Bruce Perens. 2005. The emerging economic paradigm of open source. First Monday (2005).
- 10.5555/2817915.2817918 · doi:10.5555/2817915.2817918
- Cynthia Rudin Caroline Wang and Beau Coker. 2018. The age of secrecy and unfairness in recidivism prediction. arXiv preprint arXiv:1811.00731(2018). · arXiv
- 10.1891/0886-6708.8.4.339 · DOI
- 10.2307/222715 · doi:10.2307/222715
- Richard Stallman. 2016. Why programs must not limit the freedom to run them. https://www.gnu.org/philosophy/free-sw.en.html. The Free Software Foundation (18 Nov. 2016). Accessed: 2021-07-1.
- Richard Stallman 2021. What is free software?, The Free Software Definition, Version 1.169. https://www.gnu.org/philosophy/free-sw.en.html. The Free Software Foundation (2 Feb. 2021). Accessed: 2021-07-1.
- Luke Stark, Daniel Greene, and Anna Lauren Hoffmann. 2021. Critical Perspectives on Governance Mechanisms for AI/ML Systems. In The Cultural Life of Machine Learning. Springer, 257–280.
- 10.1089/cpb.2006.9.552 · DOI
- Anselm Strauss and Juliet M Corbin. 1997. Grounded theory in practice. Sage.
- M Tiemann and Open Source Initiative. 2009. History of the OSI (Open Source Initiative). Retrieved February 4(2009), 2009.
- Marcus Tomalin Bill Byrne Shauna Concannon Danielle Saunders and Stefanie Ullmann. 2021. The practical ethics of bias reduction in machine translation: Why domain adaptation is better than data debiasing. Ethics and Information Technology(2021) 1–15. · DOI
- Joachim Van den Bergh and Dirk Deschoolmeester. 2010. Ethical decision making in ICT: discussing the impact of an ethical code of conduct. Communications of the IBIMA(2010) 1–10. · DOI · Paper page
- 10.1145/2328909.2328916 · DOI
- Robert S Weiss. 1995. Learning from strangers: The art and method of qualitative interview studies. Simon and Schuster.
- 10.1145/3488666 · DOI · Paper page
- 10.1080/23268743.2019.1642794 · DOI · Paper page
- 10.1145/563517.563464 · doi:10.1145/563517.563464
- Marty J Wolf, Keith W Miller, and Frances S Grodzinsky. 2019. On the Responsibility for Uses of Downstream Software. Computer Ethics-Philosophical Enquiry (CEPE) Proceedings 2019, 1(2019), 3.
- Robert K Yin. 2017. Case study research and applications: Design and methods. Sage publications.
- 10.1145/3338906.3338918 · DOI
Cited by
Typology of Risks of Generative Text-to-Image Models
Bird, C., Ungless, E., & Kasirzadeh, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604722 · Cited by 121
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90
Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries
Umbach, R., Henry, N., Beard, G. F., & Berryessa, C. M. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642382 · Cited by 73
Seger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., Ó hÉigeartaigh, S., Korinek, A., et al. (2023) · doi:10.2139/ssrn.4596436 · Cited by 31
Studying the Online Deepfake Community
Timmerman, B., Mehta, P., Deb, P., Gallagher, K., Dolan-Gavitt, B., Garg, S., & Greenstadt, R. (2023) · Journal of Online Trust and Safety · doi:10.54501/jots.v2i1.126 · Cited by 17
Metadata
- Authors
- Widder, D. G., Nafus, D., Dabbish, L., & Herbsleb, J.
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 2035-2046
- OpenAlex
- W4283172171
- Cited by
- 60
- References
- 15
- FWCI
- 12.33
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes. https://app.cochat.ai/papers/10.1145/3531146.3533779
- The paper
- Widder, D. G., Nafus, D., Dabbish, L., & Herbsleb, J. (2022). Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes. 2022 ACM Conference on Fairness Accountability and Transparency. https://doi.org/10.1145/3531146.3533779
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.