Understanding accountability in algorithmic supply chains
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Academic and policy proposals on algorithmic accountability often seek to understand algorithmic systems in their socio-technical context, recognising that they are produced by ‘many hands’. Increasingly, however, algorithmic systems are also produced, deployed, and used within a supply chain comprising multiple actors tied together by flows of data between them. In such cases, it is the working together of an algorithmic supply chain of different actors who contribute to the production, deployment, use, and functionality that drives systems and produces particular outcomes. We argue that algorithmic accountability discussions must consider supply chains and the difficult implications they raise for the governance and accountability of algorithmic systems. In doing so, we explore algorithmic supply chains, locating them in their broader technical and political economic context and identifying some key features that should be understood in future work on algorithmic governance and accountability (particularly regarding general purpose AI services). To highlight ways forward and areas warranting attention, we further discuss some implications raised by supply chains: challenges for allocating accountability stemming from distributed responsibility for systems between actors, limited visibility due to the accountability horizon, service models of use and liability, and cross-border supply chains and regulatory arbitrage
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 6×
“Addressing the ‘many hands’ problem is increasingly a concern in the algorithmic accountability literature (implicitly or explicitly), with proposals in recent years for accountability for algorithmic systems to operate at an organisational level [25, 40, 60, 62, 67, 75, 89, 98, 110].”
BackgroundInfluentialGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 6×
“Yet organisation-focused tools to provide information on points in the AI lifecycle (such as [39, 75]) are of limited help where information about interconnections be-tween actors is needed.”
BackgroundInfluentialRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 3×
“Addressing the ‘many hands’ problem is increasingly a concern in the algorithmic accountability literature (implicitly or explicitly), with proposals in recent years for accountability for algorithmic systems to operate at an organisational level [25, 40, 60, 62, 67, 75, 89, 98, 110].”
BackgroundAnanny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · cited 1×
“A particular difficulty, however, is that accountability is contextual [5, 12, 23, 80, 109, 110].”
BackgroundFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Veale, M., & Zuiderveen Borgesius, F. (2021) · Computer Law Review International · doi:10.9785/cri-2021-220402 · Cited by 805
Self-citationMoral Responsibility of Public Officials: The Problem of Many Hands
Thompson, D. F. (1980) · American Political Science Review · doi:10.2307/1954312 · Cited by 651
Edwards, L., & Veale, M. (2017) · doi:10.31228/osf.io/97upg · Cited by 524
Self-citationAequitas: A Bias and Fairness Audit Toolkit
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018) · arXiv · doi:10.48550/arxiv.1811.05577 · Cited by 496
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · cited 1×
“Addressing the ‘many hands’ problem is increasingly a concern in the algorithmic accountability literature (implicitly or explicitly), with proposals in recent years for accountability for algorithmic systems to operate at an organisational level [23, 39, 60, 62, 67, 75, 89, 98, 110].”
BackgroundFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · cited 3×
“Even more recent work on algorithmic accountability which seeks to address the ‘many hands’ problem through a relatively broad view of accounting for algorithmic systems is typically focused on making specific stages of system lifecycle more transparent and understandable [7, 30, 40, 75, 85] or framed around the persp…”
BackgroundExplanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388 · cited 1×
“Many academic, policy, and legislative initiatives propose impact assessments, risk assessments, and risk management mechanisms to mitigate harms of AI technologies (for example, [1, 8, 23, 28, 31, 34, 39, 43, 49, 52, 53, 56, 57, 59, 60, 63, 70, 75, 88, 89, 92, 94, 110]).”
BackgroundWhat Things Do: Philosophical Reflections on Technology, Agency, and Design
Verbeek, P.-P. (2005) · doi:10.1515/9780271033228 · Cited by 371
Accountability in a computerized society
Nissenbaum, H. (1996) · Science and Engineering Ethics · doi:10.1007/bf02639315 · Cited by 346
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · cited 5×
“A particular difficulty, however, is that accountability is contextual [5, 12, 25, 80, 109, 110].”
BackgroundInfluentialProblem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283 · cited 1×
“The ‘many hands’ problem made critical questions of identifying who framed the problem and when it was framed difficult to answer [82].”
BackgroundHutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445921 · Cited by 150 · cited 8×
“A particular difficulty, however, is that accountability is contextual [5, 12, 25, 80, 109, 110].”
BackgroundInfluentialSelf-citation(2024) · doi:10.1093/oso · Cited by 138
Gandy, O. H. (2010) · Ethics and Information Technology · doi:10.1007/s10676-009-9198-6 · Cited by 115 · cited 2×
“While literature in these fields considers issues such as the cumulative effects of systems on individuals and communities [38], they typically consider systems themselves through an organisation-centric lens – equating particular functionality (credit scoring, criminal profiling, airport…”
BackgroundDecision Provenance: Harnessing Data Flow for Accountable Systems
Singh, J., Cobbe, J., & Norval, C. (2019) · IEEE Access · doi:10.1109/access.2018.2887201 · Cited by 110
Self-citationInteractive Model Cards: A Human-Centered Approach to Model Documentation
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533108 · Cited by 101 · cited 3×
“or using a system, such as for datasheets [39] or data cards [86] to describe datasets, or model cards [29, 75] and factsheets [7] to describe model specification and capabilities.”
BackgroundMethodDemystifying the Draft EU Artificial Intelligence Act
Veale, M., & Zuiderveen Borgesius, F. (2021) · doi:10.31235/osf.io/38p5f · Cited by 91
Self-citationGürses, S., & van Hoboken, J. (2018) · The Cambridge Handbook of Consumer Privacy · doi:10.1017/9781316831960.032 · Cited by 70
Accountability in Computer Systems
Kroll, J. A. (2020) · The Oxford Handbook of Ethics of AI · doi:10.1093/oxfordhb/9780190067397.013.10 · Cited by 41 · cited 2×
“algorithmic accountability literature explicitly views accountability as a mechanism for holding actors to account for systems they are responsible for [1, 25, 61, 109].”
BackgroundMonitoring AI Services for Misuse
Javadi, S. A., Norval, C., Cloete, R., & Singh, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462566 · Cited by 21 · cited 1×
“‘Know your customer’ requirements around on-boarding for AI services (common in financial services) could help providers understand customers’ purposes and intentions [24], as can technical measures for monitoring how their services are (mis)used [53, 54] (though these only give some visibility over one or two steps i…”
BackgroundSelf-citationWhat lies beneath: transparency in online service supply chains
Cobbe, J., Norval, C., & Singh, J. (2020) · Journal of Cyber Policy · doi:10.1080/23738871.2020.1745860 · Cited by 13
Self-citation
Cited by
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 123
Auditing Work: Exploring the New York City algorithmic bias audit regime
Groves, L., Metcalf, J., Kennedy, A., Vecchione, B., & Strait, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658959 · Cited by 59
“That’s how we try to put it.”
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Law and the Emerging Political Economy of Algorithmic Audits
Terzis, P., Veale, M., & Gaumann, N. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658970 · Cited by 39
Self-citationShould Users Trust Advanced AI Assistants? Justified Trust As a Function of Competence and Alignment
Manzini, A., Keeling, G., Marchal, N., McKee, K. R., Rieser, V., & Gabriel, I. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658964 · Cited by 28
Metadata
- Authors
- Cobbe, J., Veale, M., & Singh, J.
- Venue
- 2023 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2023-06-12
- Pages
- 1186-1197
- OpenAlex
- W4367628156
- Cited by
- 115
- References
- 30
- FWCI
- 29.66
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.