CoChat
Sign Up
Cited by 96

AI auditing: The Broken Bus on the Road to AI Accountability

Birhane, Abeba, Steed, Ryan, Ojewale, Victor, Vecchione, Briana, Raji, Inioluwa Deborah·2024·2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)
DOI 10.1109/satml59370.2024.00037
Open this paper in CoChat — free account

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/satml59370.2024.00037

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 69

  1. B. Imana, A. Korolova, and J. Heidemann, "Auditing for Discrimination in Algorithms Delivering Job Ads," in Proceedings of the Web Conference 2021. Ljubljana Slovenia: ACM, Apr. 2021, pp. 3767- 3778. [Online]. Available: https://dl.acm.org/doi/10. 1145/3442381.3450077 · DOI
  2. S. Luccioni, C. Akiki, M. Mitchell, and Y. Jernite, "Stable Bias: Evaluating Societal Representations in Diffusion Models," in Thirty-Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, Nov. 2023. [Online]. Available: https://openreview.net/forum?id=qVXYU3F017
  3. P. Barlas, K. Kyriakou, O. Guest, S. Kleanthous, and J. Otterbacher, "To 'see' is to stereotype: Image tag- ging algorithms, gender recognition, and the accuracy- fairness trade-off," Proceedings of the ACM on Human- Computer Interaction, vol. 4, no. CSCW3, pp. 1-31, 2021.
  4. I. Kilovaty, "Legally cognizable manipulation," Berke- ley Tech. LJ, vol. 34, p. 449, 2019.
  5. C. Cadwalladr and E. Graham-Harrison, "Revealed: 50 million facebook profiles harvested for cambridge analytica in major data breach," The Guardian, vol. 17, no. 1, p. 22, 2018.
  6. F. Pasquale, The black box society: The secret algo- rithms that control money and information. Harvard University Press, 2015.
  7. M. Power, The Audit Society: Rituals of Verification. Oxford, New York: Oxford University Press, Oct. 1999.
  8. F. Cherry and M. Bendick, "Making it count: Discrimi- nation auditing and the activist scholar tradition," Audit studies: Behind the scenes with theory, method, and nuance, pp. 45-62, 2018.
  9. B. Vecchione, S. Barocas, and K. Levy, "Algorithmic Auditing and Social Justice: Lessons from the History of Audit Studies," arXiv:2109.06974 [cs], Sep. 2021. [Online]. Available: http://arxiv.org/abs/2109.06974 · arXiv
  10. N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramer, B. Balle, D. Ippolito, and E. Wallace, "Ex- tracting training data from diffusion models," in 32nd USENIX Security Symposium (USENIX Security 23), 2023, pp. 5253-5270.
  11. M. Wieringa, "What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability," in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 2020, pp. 1-18.
  12. A. D. Selbst, "An institutional view of algorithmic impact," Harvard Journal of Law & Technology, vol. 35, no. 1, 2021.
  13. R. Steed and A. Caliskan, "Image representations learned with unsupervised pre-training contain human- like biases," in Proceedings of the 2021 ACM con- ference on Fairness, Accountability, and Transparency, 2021, pp. 701-713.
  14. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., "Llama: Open and efficient foundation language models," arXiv preprint arXiv:2302.13971, 2023. · arXiv
  15. A. Birhane and V. U. Prabhu, "Large image datasets: A pyrrhic win for computer vision?" in 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2021, pp. 1536-1546.
  16. O. Solon, "Facial recognition's 'dirty little secret': Mil- lions of online photos scraped without consent," NBC News, vol. 12, 2019.
  17. C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, "Fairness through awareness," in Proceedings of the 3rd innovations in theoretical computer science conference, 2012, pp. 214-226.
  18. I. D. Raji and J. Buolamwini, "Actionable Auditing Revisited: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products," Communications of the ACM, vol. 66, no. 1, pp. 101-108, Dec. 2022. [Online]. Available: https://dl.acm.org/doi/10.1145/3571151 · DOI · Paper page
  19. A. Chouldechova, D. Benavides-Prado, O. Fialko, and R. Vaithianathan, "A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions," in Conference on Fairness, Accountability and Transparency. PMLR, 2018, pp. 134-148.
  20. L. Chen, A. Mislove, and C. Wilson, "Peeking beneath the hood of uber," in Proceedings of the 2015 Internet Measurement Conference, 2015, pp. 495-508.
  21. --, "An empirical analysis of algorithmic pricing on amazon marketplace," in Proceedings of the 25th international conference on World Wide Web, 2016, pp. 1339-1349.
  22. J. Buolamwini and T. Gebru, "Gender Shades: Inter- sectional Accuracy Disparities in Commercial Gender Classification," in Proceedings of the 1st Conference on Fairness, Accountability and Transparency, ser. Proceedings of Machine Learning Research, S. A. Friedler and C. Wilson, Eds., vol. 81. New York, NY, USA: PMLR, Jan. 2018, pp. 77-91. [Online]. Available: http://proceedings.mlr.press/v81/buolamwini18a.html
  23. A. Mandal, S. Little, and S. Leavy, "Gender bias in multimodal models: A transnational feminist approach considering geographical region and culture," arXiv preprint arXiv:2309.04997, 2023. · arXiv
  24. C. Wilson, A. Ghosh, S. Jiang, A. Mislove, L. Baker, J. Szary, K. Trindel, and F. Polli, "Building and auditing fair algorithms: A case study in candidate screening," in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 666-677.
  25. A. Koenecke, E. Giannella, R. Willer, and S. Goel, "Popular support for balancing equity and efficiency in resource allocation: A case study in online advertising to increase welfare program awareness," in Proceedings of the International AAAI Conference on Web and Social Media, vol. 17, 2023, pp. 494-506.
  26. P. Sapiezynski, A. Ghosh, L. Kaplan, A. Rieke, and A. Mislove, "Algorithms that "Don't See Color": Comparing Biases in Lookalike and Special Ad Audiences," in Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, Jul. 2022, pp. 609-616. [Online]. Available: http://arxiv.org/abs/1912. 07579
  27. S. U. Noble, Algorithms of Oppression: How Search En- gines Reinforce Racism. New York: NYU Press, 2018. [Online]. Available: www.jstor.org/stable/j.ctt1pwt9w5
  28. I. Ajunwa, S. Friedler, C. E. Scheidegger, and S. Venkatasubramanian, "Hiring by algorithm: pre- dicting and preventing disparate impact," Available at SSRN, 2016.
  29. J. Bandy, "Problematic machine behavior: A systematic literature review of algorithm audits," Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW1, pp. 1-34, 2021.
  30. P. Krafft, M. Young, M. Katell, J. E. Lee, S. Narayan, M. Epstein, D. Dailey, B. Herman, A. Tam, V. Guetler et al., "An action-oriented ai policy toolkit for technol- ogy audits by community advocates and activists," in Proceedings of the 2021 ACM conference on Fairness, Accountability, and Transparency, 2021, pp. 772-781.
  31. S. Jiang, R. E. Robertson, and C. Wilson, "Bias Misperceived:The Role of Partisanship and Misinformation in YouTube Comment Moderation," Proceedings of the International AAAI Confer- ence on Web and Social Media, vol. 13, pp. 278-289, Jul. 2019. [Online]. Available: https://ojs.aaai.org/index.php/ICWSM/article/view/3229
  32. E. Black, H. Elzayn, A. Chouldechova, J. Goldin, and D. Ho, "Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models," in 2022 ACM Conference on Fairness, Accountability, and Transparency. Seoul Republic of Korea: ACM, Jun. 2022, pp. 1479-1503. [Online]. Available: https: //dl.acm.org/doi/10.1145/3531146.3533204 · DOI
  33. A. Abid, M. Farooqi, and J. Zou, "Persistent Anti-Muslim Bias in Large Language Models," in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, ser. AIES '21. New York, NY, USA: Association for Computing Machinery, Jul. 2021, pp. 298-306. [Online]. Available: https: //doi.org/10.1145/3461702.3462624 · DOI · Paper page
  34. A. Birhane and V. U. Prabhu, "Large image datasets: A pyrrhic win for computer vision?" in 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Jan. 2021, pp. 1536-1546.
  35. A. Brown, A. Chouldechova, E. Putnam-Hornstein, A. Tobin, and R. Vaithianathan, "Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, ser. CHI '19. New York, NY, USA: Association for Computing Machinery, May 2019, pp. 1-12. [Online]. Available: https: //dl.acm.org/doi/10.1145/3290605.3300271 · DOI · Paper page
  36. A. Birhane, V. U. Prabhu, and E. Kahembwe, "Multimodal datasets: Misogyny, pornography, and malignant stereotypes," Oct. 2021. [Online]. Available: http://arxiv.org/abs/2110.01963 · arXiv
  37. M. Bao, A. Zhou, S. Zottola, B. Brubach, S. Desmarais, A. Horowitz, K. Lum, and S. Venkatasubramanian, "It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks," arXiv:2106.05498 [cs], Jun. 2021. [Online]. Available: http://arxiv.org/abs/2106.05498 · arXiv
  38. P. Samuelson, "Generative AI meets copyright," Sci- ence, vol. 381, no. 6654, pp. 158-161, 2023.
  39. L. Stapleton, M. H. Lee, D. Qing, M. Wright, A. Chouldechova, K. Holstein, Z. S. Wu, and H. Zhu, "Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stake- holders," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 1162-1177.
  40. E. Radiya-Dixit and G. Neff, "A Sociotechnical Audit: Assessing Police Use of Facial Recognition," in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT '23. New York, NY, USA: Association for Computing Machinery, Jun. 2023, pp. 1334-1346. [Online]. Available: https://dl.acm.org/doi/10.1145/3593013.3594084 · DOI
  41. L. Stapleton, M. H. Lee, D. Qing, M. Wright, A. Chouldechova, K. Holstein, Z. S. Wu, and H. Zhu, "Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT '22. New York, NY, USA: Association for Computing Machinery, Jun. 2022, pp. 1162-1177. [Online]. Available: https: //dl.acm.org/doi/10.1145/3531146.3533177 · DOI
  42. A. Woodruff, S. E. Fox, S. Rousso-Schindler, and J. Warshaw, "A Qualitative Exploration of Perceptions of Algorithmic Fairness," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ser. CHI '18. New York, NY, USA: Association for Computing Machinery, Apr. 2018, pp. 1-14. [Online]. Available: https://dl.acm.org/doi/10. 1145/3173574.3174230 · DOI · Paper page
  43. I. Solaiman, Z. Talat, W. Agnew, L. Ahmad, D. Baker, S. L. Blodgett, H. Daumé III, J. Dodge, E. Evans, S. Hooker, Y. Jernite, A. S. Luccioni, A. Lusoli, M. Mitchell, J. Newman, M.-T. Png, A. Strait, and A. Vassilev, "Evaluating the Social Impact of Generative AI Systems in Systems and Society," Jun. 2023. [Online]. Available: http://arxiv.org/abs/2306.05949 · arXiv
  44. D. Ramesh, V. Kameswaran, D. Wang, and N. Sambasivan, "How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT '22. New York, NY, USA: Association for Computing Machinery, Jun. 2022, pp. 1917-1928. [Online]. Available: https://dl.acm.org/doi/10.1145/3531146.3533237 · DOI
  45. H. Suresh and J. Guttag, "A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle," in Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, ser. EAAMO '21. New York, NY, USA: Association for Computing Machinery, Nov. 2021, pp. 1-9. [Online]. Available: https://dl.acm.org/doi/10.1145/3465416.3483305 · DOI · Paper page
  46. R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, "Green AI," Communications of the ACM, vol. 63, no. 12, pp. 54-63, Nov. 2020. [Online]. Available: https://dl.acm.org/doi/10.1145/3381831 · DOI · Paper page
  47. B. Rakova and R. Dobbe, "Algorithms as social- ecological-technological systems: an environmental justice lens on algorithmic audits," arXiv preprint arXiv:2305.05733, 2023. · arXiv
  48. K. Crawford, The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press, 2021.
  49. B. Gansky and S. McDonald, "CounterFAccTual: How FAccT Undermines Its Organizing Principles," in 2022 ACM Conference on Fairness, Accountability, and Transparency. Seoul Republic of Korea: ACM, Jun. 2022, pp. 1982-1992. [Online]. Available: https: //dl.acm.org/doi/10.1145/3531146.3533241 · DOI · Paper page
  50. E. P. Goodman and J. Trehu, "AI Audit Washing and Accountability," Rochester, NY, Sep. 2022. [Online]. Available: https://papers.ssrn.com/abstract=4227350
  51. M. Sloane, E. Moss, and R. Chowdhury, "A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability," Patterns (New York, N.Y.), vol. 3, no. 2, p. 100425, Feb. 2022.
  52. R. A. Calix, J. Ben-Joseph, N. Lopatina, R. Ashley, M. Gogia, G. Sieniawski, and A. Brennen, "Saisiyat Is Where It Is At! Insights Into Backdoors And Debiasing Of Cross Lingual Transformers For Named Entity Recognition," in 2022 IEEE International Conference on Big Data (Big Data), Dec. 2022, pp. 2940- 2949. [Online]. Available: https://ieeexplore.ieee.org/ document/10020403
  53. A. Warofka, "An independent assessment of the hu- man rights impact of facebook in myanmar," Facebook Newsroom, November, vol. 5, 2018.
  54. N. Diakopoulos, "Algorithmic accountability: Journal- istic investigation of computational power structures," Digital journalism, vol. 3, no. 3, pp. 398-415, 2015.
  55. P. J. Phillips, P. Grother, R. Micheals, D. M. Blackburn, E. Tabassi, and M. Bone, "Face recognition vendor test 2002," in 2003 IEEE International SOI Conference. Proceedings (Cat. No. 03CH37443). IEEE, 2003, p. 44.
  56. M. S. Lam, A. Pandit, C. H. Kalicki, R. Gupta, P. Sahoo, and D. Metaxa, "Sociotechnical Audits: Broadening the Algorithm Auditing Lens to Investigate Targeted Advertising," Proceedings of the ACM on Human-Computer Interaction, vol. 7, no. CSCW2, pp. 360:1-360:37, Oct. 2023. [Online]. Available: https://dl.acm.org/doi/10.1145/3610209 · DOI
  57. I. D. Raji, T. Gebru, M. Mitchell, J. Buolamwini, J. Lee, and E. Denton, "Saving face: Investigating the ethical concerns of facial recognition auditing," in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 2020, pp. 145-151.
  58. A. D. Selbst, D. Boyd, S. A. Friedler, S. Venkatasubramanian, and J. Vertesi, "Fairness and Abstraction in Sociotechnical Systems," in Proceedings of the Conference on Fairness, Accountability, and Transparency, ser. FAT* '19. New York, NY, USA: Association for Computing Machinery, Jan. 2019, pp. 59-68. [Online]. Available: https://dl.acm.org/doi/10.1145/3287560.3287598 · DOI · Paper page
  59. A. Xiang, "Reconciling Legal and Technical Approaches to Algorithmic Bias," Tennessee Law Review, vol. 88, no. 3, pp. 649-724, 2020. [Online]. Available: https://heinonline.org/HOL/P?h= hein.journals/tenn88&i=669
  60. A. Xiang and I. D. Raji, "On the Legal Compatibility of Fairness Definitions," Nov. 2019. [Online]. Available: http://arxiv.org/abs/1912.00761 · arXiv
  61. B. Green, "Escaping the impossibility of fairness: From formal to substantive algorithmic fairness," Philosophy & Technology, vol. 35, no. 4, p. 90, 2022.
  62. B. Laufer, J. Kleinberg, K. Levy, and H. Nissenbaum, "Strategic Evaluation," in Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, ser. EAAMO '23. New York, NY, USA: Association for Computing Machinery, Oct. 2023, pp. 1-12. [Online]. Available: https://dl.acm.org/doi/10.1145/3617694.3623237 · DOI
  63. T. Phan, J. Goldenfein, M. Mann, and D. Kuch, "Economies of virtue: the circulation of 'ethics' in big tech," Science as culture, vol. 31, no. 1, pp. 121-135, 2022.
  64. W. Boag, H. Suresh, B. Lepe, and C. D'Ignazio, "Tech worker organizing for power and accountability," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 452-463.
  65. D. G. Widder, D. Zhen, L. Dabbish, and J. Herbsleb, "It's about power: What ethical concerns do software engineers have, and what do they (feel they can) do about them?" in Proceedings of the 2023 ACM Con- ference on Fairness, Accountability, and Transparency, 2023, pp. 467-479.
  66. M. Ryan, E. Christodoulou, J. Antoniou, and K. Ior- danou, "An AI ethics 'David and Goliath': value con- flicts between large tech companies and their employ- ees," AI & Society, pp. 1-16, 2022.
  67. H. Schellmann, "Auditors are testing hiring algorithms for bias, but there's no easy fix," MIT Technology Review, February, vol. 11, p. 2021, 2021.
  68. L. Weidinger, M. Rauh, N. Marchal, A. Manzini, L. A. Hendricks, J. Mateos-Garcia, S. Bergman, J. Kay, C. Griffin, B. Bariach, I. Gabriel, V. Rieser, and W. Isaac, "Sociotechnical Safety Evaluation of Generative AI Systems," Oct. 2023. [Online]. Available: http://arxiv.org/abs/2310.11986 · arXiv
  69. A. Birhane, P. Kalluri, D. Card, W. Agnew, R. Dotan, and M. Bao, "The values encoded in machine learning research," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 173-184.

Cited by

Sorted by citations

Authors
Birhane, A., Steed, R., Ojewale, V., Vecchione, B., & Raji, I. D.
Affiliations
  • Trinity College Dublin,Mozilla Foundation and Sch. of Computer Science and Statistics,Dublin,Ireland
  • Carnegie Mellon University,Heinz College & Machine Learning Dept.,Pittsburgh,USA
  • Brown University,Ctr. for Technological Responsibility, Reimagination, & Redesign,Providence,USA
  • Data & Society,New York City,USA
  • Mozilla Foundation and University of California, Berkeley,Berkeley,USA
Venue
2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)
Publisher
IEEE
Published
2024-04-09
Pages
612-643
DOI
10.1109/satml59370.2024.00037
Cited by
96
References
23
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review