CoChat

Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts

Open access
Metcalf, Jacob, Moss, Emanuel, Watkins, Elizabeth Anne, Singh, Ranjit, Elish, Madeleine Clare · 2021 · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.

Abstract

Algorithmic impact assessments (AIAs) are an emergent form of accountability for organizations that build and deploy automated decision-support systems. They are modeled after impact assessments in other domains. Our study of the history of impact assessments shows that "impacts" are an evaluative construct that enable actors to identify and ameliorate harms experienced because of a policy decision or system. Every domain has different expectations and norms around what constitutes impacts and harms, how potential harms are rendered as impacts of a particular undertaking, who is responsible for conducting such assessments, and who has the authority to act on them to demand changes to that undertaking. By examining proposals for AIAs in relation to other domains, we find that there is a distinct risk of constructing algorithmic impacts as organizationally understandable metrics that are nonetheless inappropriately distant from the harms experienced by people, and which fall short of building the relationships required for effective accountability. As impact assessments become a commonplace process for evaluating harms, the FAccT community, in its efforts to address this challenge, should A) understand impacts as objects that are co-constructed accountability relationships, B) attempt to construct impacts as close as possible to actual harms, and C) recognize that accountability governance requires the input of various types of expertise and affected communities. We conclude with lessons for assembling cross-expertise consensus for the co-construction of impacts and building robust accountability relationships.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3442188.3445935

Cites

84 of 84 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 10

  1. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias: There's Software Used Across the Country to Predict Future Criminals. And it's Biased Against Blacks. ProPublica (May 2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
  2. K.A. Bamberger and Deirdre K Mulligan. [n.d.]. PIA requirements and privacy decision-making in US government agencies. In Privacy Impact Assessment. Springer, Dordrecht, 225--250.
  3. Geoffrey C. Bowker. 2013. Data flakes: An afterword to "raw data" is an oxymoron. Raw data" is an oxymoron (2013), 167--171.
  4. danah boyd. 2014. The Networked Nature of Algorithmic Discrimination. In Data and Discrimination: Collected Essays, Seeta Peña Gangadharan, With Virginia Eubanks, and Solon Barocas (Eds.). Open Technology Institute, 6.
  5. Robert W. Burchell, David Listokin, and William R. Dolphin. 1985. The New Practitioner's Guide to Fiscal Impact Analysis. Center for Urban Policy Research, New Brunswick, NJ.
  6. Tim Dare and Eileen Gambrill. 2016. Ethical analysis: Predictive risk models at call screening for Allegheny County. Unpublished report (2016).
  7. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for Datasets. arXiv:1803.09010 [cs] (March 2018). http://arxiv.org/abs/1803.09010 arXiv:1803.09010. · arXiv
  8. Tom Miles. 2018. U.N. investigators cite Facebook role in Myanmar crisis. Reuters (2018), 10.
  9. Judith Petts. 1999. Handbook of Environmental Impact Assessment Volume 2: Impact and Limitations. Vol. 2. Blackwell Science, Oxford.
  10. Steven Shapin, Simon Schaffer, and Thomas Hobbes. 1985. Leviathan and the air-pump: Hobbes, Boyle, and the experimental life: including a translation of Thomas Hobbes, Dialogus physicus de natura aeris by Simon Schaffer. Princeton University Press, Princeton, N.J.

Cited by

See all 196

Metadata

Authors
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C.
Affiliations
  • Data & Society Research Institute (US)
  • The Graduate Center, CUNY (US)
Venue
Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2021-03
Pages
735-746
OpenAlex
W3134214835
Cited by
195
References
79
Citations by year
2026: 61 · 2025: 40 · 2024: 27 · 2023: 26 · 2022: 29 · 2021: 12
FWCI
21.25
Type
conference paper
Language
en
Open access
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc-nd)
Funders
National Science Foundation (1704425)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Law › Digitalization, Law, and Regulation
Keywords
algorithmic impact assessment, accountability, automated decision support system, harm, accountability relationships, affected communities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.