Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Open accessThe DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Abstract
Algorithmic impact assessments (AIAs) are an emergent form of accountability for organizations that build and deploy automated decision-support systems. They are modeled after impact assessments in other domains. Our study of the history of impact assessments shows that "impacts" are an evaluative construct that enable actors to identify and ameliorate harms experienced because of a policy decision or system. Every domain has different expectations and norms around what constitutes impacts and harms, how potential harms are rendered as impacts of a particular undertaking, who is responsible for conducting such assessments, and who has the authority to act on them to demand changes to that undertaking. By examining proposals for AIAs in relation to other domains, we find that there is a distinct risk of constructing algorithmic impacts as organizationally understandable metrics that are nonetheless inappropriately distant from the harms experienced by people, and which fall short of building the relationships required for effective accountability. As impact assessments become a commonplace process for evaluating harms, the FAccT community, in its efforts to address this challenge, should A) understand impacts as objects that are co-constructed accountability relationships, B) attempt to construct impacts as close as possible to actual harms, and C) recognize that accountability governance requires the input of various types of expertise and affected communities. We conclude with lessons for assembling cross-expertise consensus for the co-construction of impacts and building robust accountability relationships.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, most proposed policies have offered little guidance on audit quality assurance issues, such as independence rules, engagement performance matters, auditor qualification, and quality control procedures [40].”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [40]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“More broadly, scholarship on algorithmic governance emphasizes that accountability requires institutionalized mechanisms for participation, feedback, and revision over time.”
From The Due Process Deficit: Auditing AI Governance in U.S. Higher Education (2026) · cites this paper as [49]
The cited source says what the claim attributes to it.
In the citing paper
“Absent such mechanisms, governance frameworks risk functioning as static or symbolic commitments rather than durable, contestable structures of authority [49, 65].”
From The Due Process Deficit: Auditing AI Governance in U.S. Higher Education (2026) · cites this paper as [49]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“One study of occupational licensing laws for teachers found that stricter occupational licensing regimes is associated with a wider distribution of student test scores [52], though others concluded that state occupational licensing requirements for teachers had no impact on students' standardized test scores [28].”
From Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance (2022) · cites this paper as [52]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“For instance, it was recurring audits that revealed that Facebook was continuing to discriminate against minority users in job and housing ads,despite earlier communications that it would no longer do so [4,38,41,52,72].”
From Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance (2022) · cites this paper as [52]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter
Barad, K. (2003) · Signs: Journal of Women in Culture and Society · doi:10.1086/345321 · Cited by 7,055
Shaping Technology, Building Society: Studies in Sociotechnical Change.
Lynch, M., Bijker, W. E., & Law, J. (1993) · Contemporary Sociology · doi:10.2307/2074370 · Cited by 3,540
(2018) · Cited by 3,510
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“The most voluble discussions of impact have focused on the “disparate impact” of algorithmic systems, in which such systems replicate, extend, or entrench systematic discriminatory social biases in the classifications they make [6, 40].”
Bijker, W. E., Hughes, T. P., & Pinch, T. (1987) · Cited by 2,957
States of Knowledge: The Co-production of Science and the Social Order
Jasanoff, S. (2004) · Cited by 2,942
Privacy as Contextual IntegrityPreprint
Nissenbaum, H. F. (2004) · SSRN Electronic Journal · Cited by 2,890
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“…processes of machine learning product development: (1) Model cards: Records of how models have been developed which are portable with the model, resolving the challenge of inadvertently repurposing a model in a risky manner when it is put to use by another development team or made open source [78].”
Analysing and Assessing Accountability: A Conceptual Framework 1
Bovens, M. (2007) · European Law Journal · doi:10.1111/j.1468-0386.2007.00378.x · Cited by 2,667
Trust in numbers: the pursuit of objectivity in science and public life
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1499 · Cited by 2,619
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 2×
“As momentum builds for algorithmic impact assessments (AIAs) such as those laid out in several promising frameworks [58, 96, 97, 102], to serve as a mechanism for addressing the unintended consequences of algorithmic systems, the question of what constitutes an “impact” in the context of an AIA has never been more imp…”
Merz, S. (2020) · Ethnic and Racial Studies · doi:10.1080/01419870.2020.1715454 · Cited by 1,870
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Of bicycles, bakelites, and bulbs: toward a theory of sociotechnical change
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4464 · Cited by 1,489
Review of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. (2020) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,473
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Changing Order: Replication and Induction in Scientific Practice
Star, S. L., & Collins, H. M. (1988) · Technology and Culture · doi:10.2307/3105095 · Cited by 1,280
Changing order: Replication and induction in scientific practice
(1987) · Journal of Economic Psychology · doi:10.1016/0167-4870(87)90014-6 · Cited by 1,114
Algorithms as culture: Some tactics for the ethnography of algorithmic systems
Seaver, N. (2017) · Big Data & Society · doi:10.1177/2053951717738104 · Cited by 1,010
Trust in numbers: The pursuit of objectivity in science and public life
Matthews, J. R. (1996) · Journal of the History of the Behavioral Sciences · doi:10.1002/(sici)1520-6696(199607)32:3<241::aid-jhbs6>3.0.co;2-x · Cited by 989
Environmental impact assessment: the state of the art
Morgan, R. K. (2012) · Impact Assessment and Project Appraisal · doi:10.1080/14615517.2012.661557 · Cited by 902
The past and future of constructive technology assessment
Schot, J., & Rip, A. (1997) · Technological Forecasting and Social Change · doi:10.1016/s0040-1625(96)00180-1 · Cited by 827
Accountability in algorithmic decision making
Diakopoulos, N. (2016) · Communications of the ACM · doi:10.1145/2844110 · Cited by 763
Social impact assessment: the state of the art
Esteves, A. M., Franks, D., & Vanclay, F. (2012) · Impact Assessment and Project Appraisal · doi:10.1080/14615517.2012.660356 · Cited by 693
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651
The social shaping of technology : how the refrigerator got its hum
MacKenzie, D. A., & Wajcman, J. (1985) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · Cited by 581
Environmental impact assessment: Retrospect and prospect
Jay, S., Jones, C., Slinn, P., & Wood, C. (2007) · Environmental Impact Assessment Review · doi:10.1016/j.eiar.2006.12.001 · Cited by 519
Reassembling the Social – An Introduction to Actor Network Theory
Lissandrello, E. (2008) · Comparative Sociology · doi:10.1163/156913308x336453 · Cited by 495
The Uses of experiment: studies in the natural sciences
(1990) · Choice Reviews Online · doi:10.5860/choice.27-5743 · Cited by 457
Of Bicycles, Bakelites, and Bulbs: Toward a Theory of Sociotechnical Change
Meikle, J. L., & Bijker, W. E. (1996) · Technology and Culture · doi:10.2307/3107114 · Cited by 421
Goodhart, C. A. E. (1984) · doi:10.1007/978-1-349-17295-5 · Cited by 349
25 Years of Feminist Empiricism and Standpoint Theory: Where Are We Now?
Intemann, K. (2010) · Hypatia · doi:10.1111/j.1527-2001.2010.01138.x · Cited by 331
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · cited 2×
“Drawing on Bovens, Maranke Wieringa has developed a definition of algorithmic accountability [111]: Algorithmic accountability concerns a networked account for a socio-technical algorithmic system, following the various stages of the system’s lifecycle.”
A Methodology for Direct and Indirect Discrimination Prevention in Data Mining
Hajian, S., & Domingo-Ferrer, J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.72 · Cited by 312
On the apparent conflict between individual and group fairness
Binns, R. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372864 · Cited by 312 · cited 1×
“…methods have demonstrated the ability to describe the former (disparate classifica-tory probabilities) in any number of important ways [1, 12, 44, 110], computational methods are less well-suited to measure the ways algorithmic systems produce and distribute the risk that people and…”
AI and Big Data: A blueprint for a human rights, social and ethical impact assessment
Mantelero, A. (2018) · Computer Law & Security Review · doi:10.1016/j.clsr.2018.05.017 · Cited by 305
Owning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics
Metcalf, J., Moss, E., & boyd, D. (2019) · Social Research: An International Quarterly · doi:10.1353/sor.2019.0022 · Cited by 278
Self-citationAfter the Neutrality Ideal: Science, Politics, and "Strong Objectivity"
Harding, S. (1992) · Social Research: An International Quarterly · Cited by 273
Participation Is not a Design Fix for Machine Learning
Sloane, M., Moss, E., Awomolo, O., & Forlano, L. (2022) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3551624.3555285 · Cited by 267
Self-citationOn the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · cited 2×
“tory probabilities) in any number of important ways [1, 12, 44, 110], computational methods are less well-suited to measure the ways algorithmic systems produce and distribute the risk that people and groups might experience as a result of these classificatory processes.”
(2018) · Cited by 230
How behavioural sciences can promote truth, autonomy and democratic discourse online
Lorenz-Spreen, P., Lewandowsky, S., Sunstein, C. R., & Hertwig, R. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0889-7 · Cited by 223
Zaloom, C. (2004) · Cultural Anthropology · doi:10.1525/can.2004.19.3.365 · Cited by 217
Privacy impact assessment: Its origins and development
Clarke, R. (2009) · Computer Law & Security Review · doi:10.1016/j.clsr.2009.02.002 · Cited by 196
Human rights and impact assessment: clarifying the connections in practice
Kemp, D., & Vanclay, F. (2013) · Impact Assessment and Project Appraisal · doi:10.1080/14615517.2013.782978 · Cited by 195
Planning theories and environmental impact assessment
Lawrence, D. P. (2000) · Environmental Impact Assessment Review · doi:10.1016/s0195-9255(00)00036-6 · Cited by 193
Algorithmic impact assessments under the GDPR: producing multi-layered explanations
Kaminski, M. E., & Malgieri, G. (2021) · International Data Privacy Law · doi:10.1093/idpl/ipaa020 · Cited by 193
Disparate Impact in Big Data Policing
Selbst, A. D. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2819182 · Cited by 192
Dismantling Environmental Racism in the USA
Bullard, R. D. (1999) · Local Environment · doi:10.1080/13549839908725577 · Cited by 191
Methods of Environmental Impact Assessment
(2001) · Routledge eBooks · doi:10.4324/9780203995709 · Cited by 185
(2012) · Law, Governance and Technology Series · doi:10.1007/978-94-007-2543-0 · Cited by 183
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Data mining for discrimination discovery
Ruggieri, S., Pedreschi, D., & Turini, F. (2010) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1754428.1754432 · Cited by 166
Algorithmic realism: expanding the boundaries of algorithmic thought
Green, B., & Viljoen, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372840 · Cited by 161
Edelman, L. B., & Talesh, S. A. (2011) · Edward Elgar Publishing eBooks · doi:10.4337/9780857938732.00011 · Cited by 135
Private Accountability in an Age of Artificial Intelligence
Katyal, S. K. (2020) · The Cambridge Handbook of the Law of Algorithms · doi:10.1017/9781108680844.004 · Cited by 135
Gandy, O. H. (2010) · Ethics and Information Technology · doi:10.1007/s10676-009-9198-6 · Cited by 115 · cited 1×
“impacts are part of a larger set of automated systems’ unintended consequences, the full complement of which Oscar Gandy argues “should be subject to routine, if not continuous assessment and regulatory control” [45].”
Algorithmic Impact Assessments under the GDPR: Producing Multi-layered ExplanationsPreprint
Kaminski, M. E., & Malgieri, G. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3456224 · Cited by 106
Bieker, F., Friedewald, M., Hansen, M., Obersteller, H., & Rost, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-44760-5_2 · Cited by 104
A governance framework for algorithmic accountability and transparency
Koene, A. R., Clifton, C., Hatada, Y., Webb, H., & Richardson, R. (2019) · Repository@Nottingham (University of Nottingham) · doi:10.2861/59990 · Cited by 104
Distributed artificial intelligence
Jagannathan, V., & Dodhiawala, R. T. (1986) · ACM SIGART Bulletin · doi:10.1145/1056563.1056571 · Cited by 84
“Accountability, Quantification, and Law
Espeland, W. N., & Vannebo, B. I. (2007) · Annual Review of Law and Social Science · doi:10.1146/annurev.lawsocsci.2.081805.105908 · Cited by 78
Procurement As Policy: Administrative Process for Machine Learning
Mulligan, D. K., & Bamberger, K. A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3464203 · Cited by 75 · cited 1×
“He notes, along with others (see [84]) that the vagaries of software procurement means that government agencies may deploy algorithmic systems without actually understanding what they do, which can”
National Environmental Policy Act
(1970) · Current History · doi:10.1525/curh.1970.59.347.46 · Cited by 72
FairML : ToolBox for diagnosing bias in predictive modeling
Adebayo, J. (2016) · Cited by 67
Combating discrimination using Bayesian networks
Mancuhan, K., & Clifton, C. (2014) · Artificial Intelligence and Law · doi:10.1007/s10506-014-9156-4 · Cited by 65
Quantifying and Reducing Stereotypes in Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. T. (2016) · OpenBU (Boston University) · doi:10.48550/arxiv.1606.06121 · Cited by 57
Avoiding sustainability trade-offs in environmental assessment
Gibson, R. B. (2013) · Impact Assessment and Project Appraisal · doi:10.1080/14615517.2013.764633 · Cited by 54
Social Constructivism in Science and Technology Studies
Lynch, M. (2016) · Human Studies · doi:10.1007/s10746-016-9385-5 · Cited by 53
High Tech, High Risk: Tech Ethics Lessons for the COVID-19 Pandemic Response
Moss, E., & Metcalf, J. (2020) · Patterns · doi:10.1016/j.patter.2020.100102 · Cited by 37
Self-citationReducing the harms associated with risk assessments
Montague, P. (2004) · Environmental Impact Assessment Review · doi:10.1016/j.eiar.2004.06.004 · Cited by 35
Development Impact: Assessment Handbook
Burchell, R. W., Listokin, D., & Dolphin, W. R. (1993) · Cited by 32
Environmental Racism and Biased Methods of Risk Assessment
Wigley, D. C., & Shrader‐Frechette, K. (1996) · University of New Hampshire Scholars Repository (University of New Hampshire at Manchester) · Cited by 32
ECONOMIC AND FISCAL IMPACT ASSESSMENT
Leistritz, F. L. (1994) · Impact Assessment · doi:10.1080/07349165.1994.9725868 · Cited by 28
Governing with Algorithmic Impact Assessments: Six ObservationsPreprint
Moss, E., Watkins, E. A., Metcalf, J., & Elish, M. C. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3584818 · Cited by 17
Self-citationDiscrimination discovery in scientific project evaluation: A case study
Romei, A., Ruggieri, S., & Turini, F. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.05.016 · Cited by 15
Remedies for Environmental Racism: A View from the Field
Cole, L. W. (1992) · Michigan Law Review · doi:10.2307/1289740 · Cited by 14
Entanglements and Exploits: Sociotechnical Security as an Analytic Framework
Goerzen, M., Watkins, E. A., & Lim, G. (2019) · USENIX Security Symposium · Cited by 8
Self-citationForman, L., & MacNaughton, G. (2016) · Impact Assessment and Project Appraisal · doi:10.1080/14615517.2016.1140995 · Cited by 7
An Analytic Challenge: Discrimination Theory in the Age of Predictive AnalyticsPreprint
Zarsky, T. (2017) · SSRN Electronic Journal · Cited by 3
Internet and the Justice System
Cerf, V. G. (2004) · Washington law review · Cited by 3
Research works printed but not linked: 10
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias: There's Software Used Across the Country to Predict Future Criminals. And it's Biased Against Blacks. ProPublica (May 2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- K.A. Bamberger and Deirdre K Mulligan. [n.d.]. PIA requirements and privacy decision-making in US government agencies. In Privacy Impact Assessment. Springer, Dordrecht, 225--250.
- Geoffrey C. Bowker. 2013. Data flakes: An afterword to "raw data" is an oxymoron. Raw data" is an oxymoron (2013), 167--171.
- danah boyd. 2014. The Networked Nature of Algorithmic Discrimination. In Data and Discrimination: Collected Essays, Seeta Peña Gangadharan, With Virginia Eubanks, and Solon Barocas (Eds.). Open Technology Institute, 6.
- Robert W. Burchell, David Listokin, and William R. Dolphin. 1985. The New Practitioner's Guide to Fiscal Impact Analysis. Center for Urban Policy Research, New Brunswick, NJ.
- Tim Dare and Eileen Gambrill. 2016. Ethical analysis: Predictive risk models at call screening for Allegheny County. Unpublished report (2016).
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for Datasets. arXiv:1803.09010 [cs] (March 2018). http://arxiv.org/abs/1803.09010 arXiv:1803.09010. · arXiv
- Tom Miles. 2018. U.N. investigators cite Facebook role in Myanmar crisis. Reuters (2018), 10.
- Judith Petts. 1999. Handbook of Environmental Impact Assessment Volume 2: Impact and Limitations. Vol. 2. Blackwell Science, Oxford.
- Steven Shapin, Simon Schaffer, and Thomas Hobbes. 1985. Leviathan and the air-pump: Hobbes, Boyle, and the experimental life: including a translation of Thomas Hobbes, Dialogus physicus de natura aeris by Simon Schaffer. Princeton University Press, Princeton, N.J.
Cited by
Trustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
“However, other researchers [138] argued that opening the black box and providing total transparency to the public does not ensure accountability.”
BackgroundAI governance: themes, knowledge gaps and future agendas
Birkstedt, T., Minkkinen, M., Tandon, A., & Mäntymäki, M. (2023) · Internet Research · doi:10.1108/intr-01-2022-0042 · Cited by 309
Defining organizational AI governance
Mäntymäki, M., Minkkinen, M., Birkstedt, T., & Viljanen, M. (2022) · AI and Ethics · doi:10.1007/s43681-022-00143-x · Cited by 285
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Putting AI ethics to work: are the tools fit for purpose?
Ayling, J., & Chapman, A. (2022) · AI and Ethics · doi:10.1007/s43681-021-00084-x · Cited by 253
Metadata
- Authors
- Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C.
- Affiliations
- Data & Society Research Institute (US)
- The Graduate Center, CUNY (US)
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 735-746
- OpenAlex
- W3134214835
- Cited by
- 195
- References
- 79
- Citations by year
- 2026: 61 · 2025: 40 · 2024: 27 · 2023: 26 · 2022: 29 · 2021: 12
- FWCI
- 21.25
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc-nd)
- Funders
- National Science Foundation (1704425)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- algorithmic impact assessment, accountability, automated decision support system, harm, accountability relationships, affected communities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.